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Abstract 

There  is  no  general  method  of  measuring  the  interoperability  of  systems  which 
accommodates  all  types  of  systems  and  interoperations.  Additionally,  no  mathematical 
method  of  describing  the  impact  of  interoperability  on  operational  effectiveness  has  been 
published.  Creating  a  general  method  of  measuring  the  interoperability  of  systems  is 
difficult  because  the  number  of  system  types  and  means  of  their  interoperation  is 
infinitely  large.  The  complexity  of  modeling  interoperations  between  all  system  types 
and  the  impossibility  of  cataloging  them  has  precluded  the  publication  of  a  general 
method  of  measuring  interoperability.  While  limited  methods  of  measuring  the 
interoperability  of  certain  types  of  systems  interoperating  in  specific  ways  have  been 
published,  these  methods  are  compartmentalized,  largely  incompatible  with  each  other, 
quickly  become  outdated  as  technology  changes,  and  produce  imprecise  measurements. 
Because  of  the  difficulty  in  creating  a  general  interoperability  measurement  method, 
other  researchers  have  relied  upon  a  problem  decomposition  approach,  effectively 
fracturing  the  problem  and  driving  them  further  from  the  solution. 

In  this  research,  a  holistic,  fundamental,  and  flexible  approach  towards  describing 
a  general  method  of  interoperability  measurement  was  taken.  The  method  applies  to  both 
collaborative  and  confrontational  interoperability.  It  models  systems  according  to  their 
interoperability-related  features  in  the  context  of  an  operational  process.  The  system 
models  are  as  abstract  or  concrete  as  desired  which  supports  a  final  interoperability 
measurement  that  is  not  limited  to  a  small  set  of  levels,  and  is  as  precise  as  desired.  A 


V 


fundamental  result  of  the  method  states  that  a  measure  of  the  similarity  of  systems 
modeled  in  this  way  is  a  measure  of  their  interoperability.  Furthermore,  if  systems 
implement  a  eonfrontational  operational  proeess  and  are  identified  and  modeled  in  the 
context  of  a  measure  of  operational  effectiveness  tied  to  that  process,  then  another 
fundamental  result  mathematically  relates  the  change  in  interoperability  of  the  systems 
with  a  change  in  the  measure  of  operational  effectiveness. 

As  a  general  method  of  measuring  interoperability  it  has  uncountable  uses, 
however  three  applications  are  given  to  illustrate  the  method.  The  first  application  shows 
how  the  method  can  be  used  to  measure  the  interoperability  of  coalition  forces  in  the 
context  of  a  multi-national  operation.  The  application  also  demonstrates  that  many  extant 
interoperability  measurement  methods  are  special  cases  of  the  more  general  method 
given  in  this  research.  The  second  application  demonstrates  the  relationship  between 
interoperability  and  operational  effectiveness  in  the  context  of  a  suppression  of  enemy  air 
defenses  (SEAD)  problem.  It  also  illustrates  that  an  interoperability  measurement  can 
motivate  system  upgrades  and  highlights  the  concept  that  friendly  systems  should  be 
directionally  interoperable  with  adversary  systems  (i.e.,  friendly  systems  should  control 
adversary  systems).  The  final  application  explains  the  time-variance  of  interoperability 
in  the  context  of  a  precision  strike  example  and  further  illustrates  the  sufficient  conditions 
for  operational  effectiveness.  More  applications  are  proposed  in  the  areas  of  non¬ 
technical  interoperability,  cross-domain  interoperability,  and  international 
interoperability.  Finally,  observing  that  the  method  in  this  dissertation  measures  direct 
interoperability  of  systems,  further  research  is  proposed  in  the  area  of  indirect 
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interoperability,  noting  that  a  system  may  not  directly  impact  adjacent  systems,  but  might 
strongly,  and  indirectly,  impact  distant  systems. 
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Definitions 


Architecture  “The  structure  of  components,  their  relationships,  and  the  principles  and 
guidelines  governing  their  design  and  evolution  over  time.”  (DoD,  2007a) 

Architecture  Framework  “Guidance  and  rules  for  structuring,  classifying,  and  organizing 
architectures.”  (Ibid) 

Blue  System  “A  U.S.  or  allied  system;  a  friendly  system.” 

Capability  “The  ability  to  execute  a  specified  course  of  action.  (A  capability  may  or  may 
not  be  accompanied  by  an  intention.)”  (JP  1,  2007) 

Character  “A  feature,  trait,  attribute,  or  characteristic.” 

Classification  “A  taxonomy.” 

Collaborative  Interoperability  “The  interoperability  between  friendly  (blue)  systems.” 

Confrontational  Interoperability  “The  interoperability  between  friendly  (blue)  and 
adversary  (red)  systems.” 

Confrontational  Operational  Process  “An  operational  process  implemented  by  two  or 
more  opposing  sets  of  systems.” 

Contextual  Interoperability  Measurement  “A  measure  of  the  interoperability  of  two 
systems  whose  instantiations  have  been  aligned  with  at  least  one  other  system 
possessed  of  one  or  more  different  characters  not  possessed  by  the  original  two 
systems.” 

Diagnostic  Character  “A  character  which  distinguishes  one  taxon  from  related  taxa.” 

Diametric  Measure  of  Operational  Effectiveness  “A  measure  of  operational  effectiveness 
written  as  a  pair  which  relates  the  effectiveness  of  the  blue  systems  to  the  lack  of 
effectiveness  of  the  red  systems.” 

Directional  Interoperation  “An  interoperation  that  occurs  from  System  A  to  System  B, 
but  not  vice  versa.” 

Effect  “A  change  to  a  condition,  behavior,  or  degree  of  freedom.”  (JP  1,  2007) 
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Interoperability  “The  ability  of  systems,  units,  or  forces  to  provide  services  to  and  accept 
services  from  other  systems,  units,  or  forces  and  to  use  the  services  so  exchanged 
to  enable  them  to  operate  effectively  together.”  (JP  1-02,  2008) 

Interoperability  Character  “Short  for  System  Interoperability  Character.  A  character 
which  describes  how  a  system  provides  and  accepts  services  from  another 
system.” 

Interoperability  Character  State  “A  qualitative  or  quantitative  instance  of  a  system 
character;  a  sub-division  of  a  system  character.” 

Interoperability  Function  “A  similarity  function  which  takes  a  pair  of  system 

instantiations  as  its  arguments,  has  a  range  of  [0,1],  rewards  for  shared  characters 
and  optionally  penalizes  for  unshared  characters,  and  gives  a  greater  reward  to 
system  pairs  whose  shared  characters’  states  have  a  “better”  value.” 

Interoperability  Measurement  “A  measure  of  the  interoperability  of  two  or  more  systems, 
or  in  other  words,  a  measure  of  the  similarity  of  two  or  more  systems  instantiated 
with  interoperability  characters.” 

Maturity  Model  “A  model  which  describes  the  stages  through  which  a  process 
progresses.”  (DoD,  1998) 

Measurement  “The  assignment  of  numbers  to  properties  or  events  in  the  real  world  by 
means  of  an  objective  empirical  operation,  in  such  a  way  as  to  describe  them.” 
(Finkelstein  &  leaning,  1984) 

Measure  of  Effectiveness  “A  criterion  used  to  assess  changes  in  system  behavior, 

capability,  or  operational  environment  that  is  tied  to  measuring  the  attainment  of 
an  end  state,  achievement  of  an  objective,  or  creation  of  an  effect.  Also  called 
MOE.”(JP  1-02,  2008) 

Measure  of  Operational  Effectiveness  “An  MOE  associated  with  an  operational  process 
which  is  used  to  assess  changes  in  the  production  of  a  desired  operational  effect. 
Also  called  MoOE.” 

Natural  Character  “A  character  which  is  not  confounded  with  another  character.” 

Numerical  Taxonomy  “The  grouping  by  numerical  methods  of  taxonomic  units  into  taxa 
on  the  basis  of  their  character  states.”  (Semple  &  Steele,  2003) 

Operation  “1.  A  military  action  or  the  carrying  out  of  a  strategic,  operational,  tactical, 

service,  training,  or  administrative  military  mission.  2.  The  process  of  carrying  on 


combat,  including  movement,  supply,  attaek,  defense,  and  maneuvers  needed  to 
gain  the  objectives  of  any  battle  or  campaign.”  (JP  1-02,  2008) 

Operational  Advantage  “The  eondition  in  whieh  a  set  of  friendly  (blue)  systems  enjoy 
direetional  interoperability  over  a  set  of  adversary  (red)  systems.” 

Operational  Process  “A  series  of  activities  and  decisions,  logically  sequenced,  whieh 
when  executed  achieve  a  desired  effect.” 

Performance  Enhanced  Instantiation  “A  system  instantiation  with  a  performanee  overlay 
(e.g.,  eost,  effieiency,  throughput,  rate,  ete.)” 

Pure  Interoperability  Measurement  “A  measure  of  the  interoperability  of  two  systems 
whose  instantiations  are  aligned  only  with  themselves.” 

Red  System  “An  enemy  or  adversary  system.” 

Self-Interoperability  “A  type  of  interoperation  which  originates  at  a  system,  exits  the 
system  boundary,  and  then  is  aecepted  back  through  the  boundary.” 

Similarity  Function  “A  funetion  used  to  measure  the  resemblance  of  two  or  more  system 
instantiations.  The  eonverse  of  a  distance  (dissimilarity  funetion).” 

System  “An  entity  whieh  is  composed  of  at  least  two  elements  and  a  relation  that  holds 
between  eaeh  of  its  elements  and  at  least  one  other  element  in  the  set”  (Ackoff, 
1971) 

System  Characterization  “A  set  of  system  instantiations  which  have  been  aligned  with 
each  other.” 

System  Identification  “The  assoeiation  of  a  system  with  the  aetivities  and  decisions  of  an 
operational  process.” 

System  Instantiation  “A  sequence  of  eharacter  states  which  models  a  system.” 

System  Similarity  “A  measure  of  the  resemblanee  of  two  systems  made  by  providing  two 
aligned  system  instantiations  representing  those  systems  as  the  arguments  to  a 
similarity  funetion.” 

Taxon  “(plural  taxa)  A  taxonomic  grouping.” 

Taxonomy  “An  orderly  grouping  of  systems  into  taxa  according  to  their  characters.” 
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Time  Variant  Interoperability  Measurement  “A  set  of  interoperability  measurements 
associated  with  a  set  of  time  periods.” 
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Notation 


An  individual  system 

s 

A  set  of  n  systems  s. 

Ci 

A  system  character  state 

c 

A  set  of  n  system  character  states  c. 

X. 

A  system  character 

X 

A  set  of  n  system  characters  x.  (called  the  Characterization  of  iS ) 

Xj. 

A  set  of  “transmit”  system  characters 

A  set  of  “receive”  system  characters 

An  instantiation  of  s. 

(7rj. 

The  “transmit”  portion  of  C- 

The  “receive”  portion  of  (J. 

A  blue  system  instantiation 

A  red  system  instantiation 

2 

A  set  of  n  aligned  system  instantiations  a.  (called  the  Instantiation  of  5^ ) 

A  set  of  aligned  blue  system  instantiations 

A  set  of  aligned  red  system  instantiations 

/ 

An  interoperability  function 

^ij 

The  directional  interoperability  measurement  from  s.  to  Sj 

M 

An  interoperability  measurement 

0 

A  diametric  measure  of  operational  effectiveness 

0, 

Blue  operational  effectiveness 

0, 

Red  operational  effectiveness 

G 

An  interoperability  graph 

V(G) 

The  vertex  set  ( iS )  of  an  interoperability  graph 

E{G) 

The  edge  set  of  an  interoperability  graph 

Sim 

A  similarity  function 
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INTEROPERABILITY  MEASUREMENT 


I.  Introduction 

Developing  and  applying  precise  measurements  in  an  area  as  multidimensional 
and  complex  as  interoperability  is  difficult.  However,  measuring,  assessing,  and 
reporting  interoperability  in  a  visible  way  is  essential  to  setting  the  right  priorities. 

— M.  Kasunic  &  W.  Anderson 


1. 1  Overview 

In  2004,  then  Secretary  of  Defense,  Donald  Rumsfeld,  said  “we’ve  put  a 
premium. .  .on  interoperability.”  That  same  year,  the  Department  of  Defense  (DoD)  hired 
a  Federally  Funded  Research  and  Development  Center  (FFRDC),  the  Camegie-Mellon 
University  Software  Engineering  Institute  (CMU-SEI),  to  research  and  report  on  the  state 
of  the  practice  in  interoperability  measurement.  They  responded  that  measuring 
interoperability  is  difficult  yet  essential  (see  quote  at  chapter  head).  (Kasunic  & 

Anderson,  2004:vii)  Unfortunately,  they  also  wrote  that  “despite  laudable  case-by-case 
efforts,  there  is  today  no  method  for  tracking  interoperability  on  a  comprehensive  or 
systematic  basis.”  (Kasunic  &  Anderson,  2004:ix)  In  fact,  CMU-SEI  highlighted  one 
extant  interoperability  measurement  model,  published  nearly  a  decade  earlier,  as  the  state 
of  the  practice  and  called  it  “immature.”  (Ibid)  There  has  been  little  change  to  the  state  of 
the  practice  since. 

1.2  Uniqueness  and  Substantiality  of  Research 

This  research  presents  an  inaugural  general  method  of  quantitatively  measuring 
the  interoperability  of  a  heterogeneous  set  of  systems.  It  overcomes  the  weaknesses  of 
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extant  methods  (Chapter  2)  which,  1)  limit  their  scope  to  specific  types  of  systems  and 
interoperability,  2)  generally  qualitatively  bin  systems  into  a  limited  number  of 
interoperability  levels,  resulting  in  an  imprecise  measurement,  3)  do  not  provide  a  means 
of  correlating  interoperability  to  operational  effectiveness  in  context  of  a  confrontational 
operational  process,  4)  restrict  themselves  to  interoperability  attributes  which  can  become 
outdated,  5)  limit  themselves  to  collaborative  interoperability  and  do  not  recognize  the 
confrontational  interoperability  between  opposing  systems,  units,  or  forces,  and  6)  do  not 
describe  appropriate  extensions  to  the  DoD  Architecture  Framework  (DoDAF)  which 
would  facilitate  the  use  of  existing  architecture  descriptions  in  performing 
interoperability  measurement.  Noting  that  “everything  in  the  world  can  be  expressed  as  a 
system,”  (Guan,  et.  ah,  2008)  this  research  uniquely  provides  a  general  method  of 
measuring  the  interoperability  of  many  different  types  of  entities,  described  as  a 
heterogeneous  set  of  systems  (e.g.,  coalitions,  technological  systems,  organizations, 
cultures,  political  philosophies,  languages,  people,  and  religions,  among  others), 
experiencing  a  wide  variety  of  types  of  interoperations  (e.g.,  enterprise,  doctrine,  force, 
joint,  logistics,  operational,  semantic,  and  technical,  among  others).  Indeed,  the  method 
improves  upon  existing  stoplight  models,  maturity  models,  interoperability  attribute 
models,  and  frameworks  by  applying  those  models’  descriptions  of  system  types  and 
interoperability  hierarchies,  levels,  and  attributes  towards  the  creation  of  a  foundational 
theory  and  general  method  of  interoperability  measurement.  The  method  of  this  research 
is  flexible  and  allows  systems  and  their  interoperations  to  be  defined  at  any  level  of 
abstraction,  resulting  in  interoperability  measurements  which  are  not  limited  to  a  small 
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set  of  possible  values,  but  are  as  precise  as  desired.  The  method  includes  proposed 
extensions  to  DoDAF  which  allow  an  architecture  to  be  better  used  in  interoperability 
analysis  and,  more  importantly,  interoperability-based  operational  effectiveness  analysis. 
The  method  recognizes  that  interoperability  is  not  an  end  unto  itself,  but  that  it  facilitates 
operational  advantage  and  effectiveness.  Finally,  the  method  introduces  the  important 
concept  of  confrontational  interoperability,  and  mathematically  correlates  the  impact  of 
interoperability  on  operational  effectiveness. 

A  brief  background  on  interoperability  is  given  next,  followed  by  a  specific 
statement  of  the  research  problem  and  associated  hypothesis  and  research  goals.  The 
interoperability  measurement  method  is  then  previewed  and  limitations  of  and 
assumptions  pertinent  to  the  method  are  given.  Chapter  1  concludes  with  an  overview  of 
the  structure  of  the  dissertation. 

1.3  Background 

Interoperability  has  been  an  important  and  widely  discussed  topic  over  the  past 
decade,  and  continues  to  be  so,  especially  within  the  Department  of  Defense  (DoD) 

(Ford,  et.  al.,  2007b).  A  search  of  thirty  years  of  definitions  and  types  of  interoperability 
(Appendices  Al  and  A2)  indicates  the  recent  surge  in  popularity  of  the  subject  (Figure  1). 
The  survey  of  interoperability  types  and  definitions  revealed  that  interoperability,  as  a 
research  area,  is  broad  with  at  least  a  thousand  academic  papers  written  on  the  topic.  The 
oldest  definition  found  (first  published  in  1977)  is  still  one  of  the  most  popular  and  is  the 
official  definition  given  in  Joint  Publication  1-02,  DoD  Dictionary  of  Military  and 
Associated  Terms,  “the  ability  of  systems,  units,  or  forces  to  provide  services  to  and 
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accept  services  from  other  systems,  units,  or  forces  and  to  use  the  services  so  exchanged 
to  enable  them  to  operate  effectively  together.”  (2008)  This  definition,  while  not  perfect 
(e.g.,  it  limits  the  object  of  an  interoperation  to  a  service  and  does  not  address  the 
confrontational  interoperability  of  adversarial  forces),  is  adopted  in  this  research  because 
it  1)  infers  that  interoperation  occurs  between  many  types  of  entities  (e.g.,  systems,  units, 
or  forces),  2)  describes  interoperability  as  a  relationship  between  those  entities,  3)  implies 
that  interoperation  is  an  exchange,  4)  infers  that  interoperation  requires  a  “provider”  and 
an  “acceptor,”  and  5)  explains  that  interoperation  enables  effective  operation.  These 
important  concepts  permeate  and  are  foundational  to  this  research. 


■  Interoperability  Definitions  ■  Interoperability  Types 


Figure  1.  Popularity  of  interoperability  research  as  indicated  by  the  number  of  definitions  and 
types  introduced  over  the  past  thirty  years  (adapted  from  Ford,  et.  al.,  2007b) 

It  is  important  to  understand  that  interoperability  occurs  in  collaborative  and  non- 

collaborative  (confrontational)  ways.  For  example,  a  computer  network  consists  of  a  set 

of  systems  working  in  a  collaborative  fashion  to  provide  and  accept  information  and 


4 


services  from  each  other.  Collaborative  interoperability  is  the  most  commonly 
understood  type  of  interoperability  today.  However,  a  second  type  of  non-collaborative 
interoperability,  called  confrontational  interoperability,  is  critical  in  both  military  and 
non-military  domains.  Confrontational  interoperability  occurs  when  sets  of  opposing 
systems  attempt  to  control  each  other  (i.e.,  a  jammer’s  attempt  to  degrade  an  enemy 
communication  system’s  effectiveness  or  a  negotiation  team’s  attempt  to  swing  a  deal  in 
their  favor).  While  collaborative  interoperability  has  been  previously  researched, 
confrontational  interoperability  has  not,  and  is  introduced  in  this  research.  An 
understanding  of  this  new  topic  of  confrontational  interoperability  is  critical  to  being  able 
to  relate  interoperability  to  operational  effectiveness. 

Joint  Vision  2020  clearly  states  the  importance  of  interoperability  to  the  DoD: 
“interoperability  is  the  foundation  of  effective  joint,  multinational,  and  interagency 
operations.”  (2006:15)  In  fact,  sixty  joint  publications  cataloged  in  the  Joint  Doctrine, 
Education  and  Training  Electronic  Information  System  (JDEIS)  mention  interoperability. 
(2008)  The  recent  Chief  of  Staff  of  the  US  Air  Force  stated  that  he  takes  every 
opportunity  “to  highlight  the  significant  advantages  of  interoperable  equipment  and 
systems  with. .  .joint  and  coalition  partners,”  that  he  looks  for  “interoperability 
opportunities  with. .  .coalition  and  allied  partners,”  and  he  recognizes  that 
“interoperability  is  cultural,  as  well  as  technical.”  (Moseley,  2007)  In  light  of  the 
importance  of  interoperability  and  noting  that  “management  must  be  able  to  measure 
what  they  wish  to  change,”  (Kasunic  &  Anderson,  2004:16)  a  comprehensive  method  for 
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measuring  collaborative  and  confrontational  interoperability  and  associating  it  with 
operational  effectiveness  is  needed. 

1.4  Problem  Statement 

How  can  collaborative  and  confrontational  interoperability  be  measured,  and  how 
can  the  effectiveness  of  an  operational  process  be  improved  by  first  measuring,  then 
changing  the  interoperability  of  a  heterogeneous  set  of  systems  implementing  the 
process? 

1.5  Hypothesis  and  Research  Objectives 

It  is  hypothesized  that  interoperability  of  a  heterogeneous  set  of  systems 
implementing  an  operational  process  can  be  measured  and  that  there  is  a  relationship 
between  that  interoperability  measurement  and  measures  of  effectiveness  associated  with 
the  process.  To  confirm  the  hypothesis,  the  following  research  objectives  are  pursued. 

1)  How  are  operational  processes  modeled  and  what  are  appropriate  measures  of 
operational  effectiveness  for  a  process? 

2)  How  are  systems  implementing  an  operational  process  identified,  modeled, 
and  classified?  Specifically,  what  characteristics,  features,  attributes,  or  traits 
of  a  system  are  important  if  the  interoperability  of  systems  is  to  be  measured? 
What  represents  a  common  framework  of  system  interoperability 
characteristics? 

3)  How  can  the  interoperability  of  systems  be  measured?  What  is  an  acceptable 
measurement  according  to  accepted  metrological  standards? 
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4)  What  is  the  relationship  between  operational  effectiveness  and 
interoperability? 

5)  What  is  the  role  of  architecture  in  interoperability  measurement?  Can  a 
Department  of  Defense  Architecture  Framework  (DoDAF)  architecture  be 
used  and/or  extended  to  source  and  store  information  required  for 
interoperability  measurement? 

6)  Demonstrate  the  interoperability  measurement  method  by  application. 

1.6  Method  Preview 

Systems  representing  both  friendly  (blue)  and  adversary  (red)  forces  implement 
the  activities  and  decisions  of  an  operational  process  and  can  be  modeled  as  a  sequence 
of  states  of  system  characteristics.  If  strictly  interoperability-related  characteristics  are 
used  to  model  systems,  then  a  fundamental  result  is  obtained — that  a  measure  of  the 
similarity  of  a  pair  of  system  models  is  a  measure  of  their  associated  systems’ 
interoperability.  This  constitutes  a  general  method  of  measuring  the  interoperability  of 
systems  implementing  both  collaborative  and  confrontational  operational  processes. 

The  research  is  taken  one  step  further  however,  and  it  is  shown  that  given  a 
measure  of  operational  effectiveness  for  a  confrontational  operational  process,  another 
fundamental  result  is  obtained  which  states  that  if  all  systems  and  system  interoperability 
features  pertinent  to  the  confrontational  operational  process  are  completely  modeled,  then 
friendly  (blue)  systems  have  operational  advantage  over  adversary  (red)  systems  if  the 
blue  systems  are  more  directionally  interoperable  with  red  systems  than  vice  versa.  In 
other  words,  blue  is  able  to  control  red  (and  prevent  red  from  reciprocating)  if  blue  is 
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more  directionally  interoperable  with  red  than  red  is  with  blue.  This  important  result 
allows  an  interoperability  measurement  to  be  related  to  operational  effectiveness. 

1.7  Assumptions/Limitations 

The  interoperability  measurement  method  presented  in  Chapter  3  is  applicable  to 
both  military  and  non-military  scenarios.  However,  in  this  research,  it  is  purposefully 
focused  on  military  applications  (Chapter  4).  Those  wishing  to  measure  the  collaborative 
(non-confrontational)  interoperability  of  non-military  systems  or  use  the  method  in 
blue/blue  situations  are  encouraged  to  rely  heavily  on  the  Performance  Enhanced 
Instantiation  section  (3.4.5)  vice  the  Interoperability  Impact  on  Operational  Effectiveness 
section  (3.6)  and  to  reference  the  discussions  on  indirect  interoperability  and  blue-to-blue 
impact  on  operational  effectiveness  presented  in  the  Future  Research  section  (5.3)  of 
Chapter  5.  Also,  as  the  method  is  model-based,  any  resultant  interoperability 
measurement  or  operational  effectiveness  assessment  should  be  accepted  at  the  same 
level  of  accuracy  and  precision  as  the  model  which  generated  it.  Indeed,  an  imprecisely 
built  model  of  a  set  of  systems  and  their  interoperations  can  result  not  only  in  an 
imprecise  interoperability  measurement,  but  possibly  inaccurate  analysis.  The  accuracy 
of  the  application  of  the  Interoperability  Impact  on  Operational  Effectiveness  axiom 
especially  depends  upon  the  accuracy  and  precision  of  the  initial  system  models. 

1.8  Dissertation  Overview 

The  next  chapter  contains  a  review  of  seminal  and  recent  publications 
underpinning  all  aspects  of  the  interoperability  measurement  method  presented  in 
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Chapter  3.  Chapter  4  uses  the  method  in  several  applieations,  demonstrating  that  1) 
extant  maturity  model  methods  of  measuring  interoperability,  sueh  as  the  Organizational 
Interoperability  Maturity  Model,  ean  be  derived  from  the  more  general  method  of  this 
dissertation,  2)  many  types  of  interoperability  can  be  measured,  such  as  coalition  and 
confrontational  interoperability,  3)  the  Interoperability  Impact  on  Operational 
Effectiveness  axiom  provides  a  sufficient  condition  for  relating  interoperability  to 
operational  advantage  or  effectiveness,  and  4)  interoperability  is  often  time  variant. 
Chapter  5  summarizes  and  makes  recommendations  for  application  of  the  dissertation 
method  and  proposes  future  research  based  upon  the  dissertation  findings.  Augmenting 
the  body  of  the  dissertation  are  multiple  appendices  which  provide  additional  original 
research  results  and  analysis  which  have  not  been  included  in  the  body  of  the  dissertation 
in  order  to  optimize  flow  and  readability  of  the  document. 
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2.  Literature  Review  and  Analysis 

The  quest  for  interoperability  is  not  new, 
but  it  has  never  been  so  important. — D.  Alberts  &  R.  Hayes 

2.1  Overview 

Current  approaches  to  interoperability  measurement,  numerical  (mathematical) 
taxonomy,  and  system  classification  are  the  starting  point  and  the  basis  for  development 
of  the  interoperability  measurement  method  in  Chapter  3.  Important  publications,  both 
historical  as  well  as  recent,  in  these  areas  and  others  are  surveyed  and  analyzed  for  their 
significant  contributions.  A  summary  of  work  in  the  supporting  topic  areas  of  process 
modeling,  operational  effectiveness  measurement,  and  architecture  is  also  presented.  An 
analysis  of  all  these  publications  is  given  which  highlights  key  concepts  pertinent  to 
interoperability  measurement  and  its  relation  of  interoperability  to  operational 
effectiveness.  Current  gaps  in  research  are  highlighted  and  discussed. 

2.2  Interoperability  Measurement 

Few  papers  have  been  published  specifically  on  interoperability  measurement. 
Allowing  for  a  very  broad  definition  of  the  term  “interoperability  measurement,” 
approximately  a  dozen  papers  have  been  written  on  the  topic — most  published  within  the 
decade.  All  limit  themselves  to  discussions  of  collaborative  interoperability  with  none 
addressing  confrontational  interoperability.  Those  proposing  a  new  interoperability 
measurement  method  or  an  extension/improvement  to  an  existing  method  include 
(LaVean,  1980;  Mensh,  et.  ah,  1989;  Amanowicz  &  Gajewski,  1996;  DoD,  1998;  Leite, 
1998;  Clark  &  Jones,  1999;  Hamilton,  et.  ah,  2002;  Alberts  &  Hayes,  2003;  Fewell  & 
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Clark,  2003;  NATO,  2003;  Tolk,  2003;  Tolk  &  Muguira,  2003;  Stewart,  et.  al.,  2004, 
2005;  Kingston,  et.  al.,  2005;  Schade,  2005;  Ford,  et.  al.,  2007a,  2008b).  Other  papers 
offer  analyses  of  existing  methods  (Sutton,  1999;  Brownsword,  et.  al.,  2004;  Kasunic  & 
Anderson,  2004;  Morris,  et.  al.,  2004;  Ford,  et.  al.,  2007b).  Each  published  method  can 
be  classified  as  maturity-  (leveling)  or  non-maturity  model-based  and  generally  is 
applicable  to  only  one  system  and  interoperability  type.  Ford,  et.  al.  provide  a  detailed 
survey  of  the  aforementioned  interoperability  methods  (Figure  2)  in  a  separate  paper 
(2007b)  which  is  excerpted  and  augmented  with  more  detail  in  Appendices  A3  and  A4. 


Figure  2.  Chronology  of  published  interoperability  measurement  methods  (non-maturity  model- 
based  methods  in  italics,  maturity-based  methods  in  boldface) 

2.2.1  Maturity  Model  and  Other  Leveling  Methods 

The  US  Air  Force  developed  the  maturity  model  concept  through  a  grant  to  the 
Carnegie  Mellon  Software  Engineering  Institute  (CMU-SEI)  in  1987.  (Humphrey  & 
Sweet,  1987)  Although  the  maturity  model  concept,  which  describes  the  stages  through 
which  a  process  progresses  (DoD,  1998:2-1),  was  originally  designed  as  a  management 
tool  to  assess  contractor  software  engineering  ability,  it  was  adopted  in  1998  by  the 
MITRE  Corporation  as  the  basis  of  the  first  maturity  model-based  interoperability 
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measurement  method  called  Levels  of  Information  System  Interoperability  (LISI)  (Figure 
3).  (Ibid)  This  method  was  eventually  formalized  and  mandated  in  CJCSI  6212. OIB 
Interoperability  and  Supportability  of  Information  Technology  and  National  Security 
Systems  (2000)  but  later  deleted  as  a  requirement  in  the  2006  issue  of  the  same  document. 
In  the  LISI  model,  maturity  was  represented  by  thresholds  of  interoperability  capability 
which  defined  measurable  levels  of  interoperability.  (DoD,  1998)  From  1998  to  2006, 
LISI  was  the  template  for  numerous  maturity  model  and  maturity  model-like  (leveling) 
interoperability  measurement  models  designed  to  measure  both  information  and  non¬ 
information  system  interoperability  such  as  the  Organizational  Interoperability  Model  for 
C2  (OIM)  (Clark  &  Jones,  1999;  Clark  &  Moon,  2001;  Fewell  &  Clark,  2003),  the 
Network  Centric  Warfare  Maturity  Model  (NCW)  (Alberts  &  Hayes,  2003),  the  Levels  of 
Conceptual  Interoperability  Model  (LCIM)  (Tolk  &  Muguira,  2003),  Layers  of  Coalition 
Interoperability  (LCI)  (Tolk,  2003),  NATO  C3  Technical  Architecture  Reference  Model 
for  Interoperability  (NMI)  (NATO,  2003),  the  Non-Technical  Interoperability  Framework 
(NTI)  (Stewart,  et.  ah,  2004),  the  Organizational  Interoperability  Agility  Model  (OIAM) 
(Kingston  et.  ah,  2005),  and  a  modification  of  the  NATO  Interoperability  Directive  (NID- 
revised)  (Schade,  2005).  As  can  be  inferred  from  the  titles  of  these  models,  none  are 
generalized  models  of  interoperability,  but  each  is  designed  to  address  a  specific  type  of 
system  and  interoperability. 
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Figure  3.  LISI  interoperability  maturity  model  (DoD,  1998) 

The  maturity  model  (leveling)  interoperability  measurement  approach  defines  a 
basic  set  of  interoperability  maturity  levels  (usually  five),  listed  as  rows  in  the  model, 
defined  by  a  set  of  attributes  (usually  three  or  four,  but  sometimes  only  one)  listed  as 
columns  in  the  model  (Figure  3).  (DoD,  1998)  Thus,  the  range  of  interoperability 
measurements  is  generally  limited  to  an  integer  from  zero  to  the  number  of  levels 
(usually  five).  This  limited  measurement  range  is  one  of  the  weaknesses  of  the  maturity 
model  interoperability  measurement  approach  as  there  is  generally  not  enough  fidelity  in 
the  measurement  to  support  design,  reliability,  maintainability,  interoperability,  or 
affordability  analyses  of  the  systems  under  measurement.  Indeed,  two  systems  sharing 
the  same  LISI  defined  interoperability  level  may  not  have  any  real  ability  to  interoperate. 
(CMU,  2008)  For  example,  two  systems  both  classified  as  level  2c  may  still  not  be  able 
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to  share  information  because  the  level  calls  for  the  ability  to  create  documents,  briefings, 
pictures,  maps,  spreadsheets,  and  databases,  yet  does  not  specify  the  application  to 
generate  these  files,  nor  does  it  specify  the  format  the  files  are  to  be  saved  in. 
Interestingly,  the  strength  of  the  maturity  model  is  not  in  providing  an  interoperability 
measurement,  but  in  its  ability  to  facilitate  a  measurement;  in  other  words,  to  portray  a 
qualitative  framework  for  describing  the  types  of  attributes  impacting  interoperability  for 
different  types  of  systems  and  interoperability.  For  example,  the  LISI  model  describes 
four  top-level  attributes  (Policy  &  Procedure,  Application,  Infrastructure,  Data)  and  at 
least  five  sub-attributes  for  each.  (DoD,  1998)  Published  maturity  model  (leveling) 
methods  apply  strictly  to  collaborative  interoperability.  As  will  be  shown  in  Chapter  4, 
the  maturity  model  and  other  leveling  methods  are  special  cases  of  the  more  general 
interoperability  measurement  method  developed  in  Chapter  3. 

2.2.2  Non-Leveling  Methods 

Non-maturity  model-based  interoperability  measurement  methods  are  a  much 
more  diverse  group  and,  as  a  whole,  generally  pre-date  the  maturity  model-based 
methods.  Like  the  maturity  model  methods,  they  are  not  generalized  methods  of 
measuring  interoperability,  but  specialized  to  a  particular  type  of  system  or 
interoperability.  The  earliest  known  model,  the  Spectrum  of  Interoperability  Model 
(SoIM),  was  designed  as  a  program  management  tool  and  defined  seven  levels  of 
interoperability  for  technical  systems.  (LaVean,  1980)  Nearly  a  decade  later,  Mensh,  et. 
ah,  published  their  mission-based  Quantification  of  Interoperability  Methodology  (QoIM) 
which  assigned  a  measure  of  effectiveness  (MOE)  logic  equation  to  each  of  seven 
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interoperability-related  eomponents.  (1989)  Their  noteworthy  contributions  are  1) 
associating  interoperability  measurement  with  a  mission  and  2)  relating  interoperability 
to  measures  of  effectiveness  via  a  discrete  event  simulation.  Seven  years  later, 
Amanowicz  &  Gajewski  made  the  important  observation  that  the  distance  between 
systems  described  in  n  dimensional  space,  according  to  their  features,  was  a  measure  of 
interoperability.  (1996)  They  drew  heavily  from  Florek,  et.  al.,  (1951)  who,  hailing  from 
Wroclaw,  Poland,  developed  a  parallel  to  numerical  taxonomy  called  Wroclaw 
Taxonomy  which  relied  upon  both  the  Czekanowski  coefficient  (Czekanowski,  1913)  and 
graphical  techniques  such  as  dendrograms.  (Sneath  &  Sokal,  1973)  Concurrent  with  the 
publication  of  LISI  was  the  Interoperability  Assessment  Methodology  (lAM),  which 
provided  an  eclectic  mix  of  interoperability  attributes  and  assorted  equations  applied  by  a 
flowcharted  interoperability  assessment  process.  (Leite,  1998)  Hamilton,  et.  al., 
criticized  LISI  as  being  too  complex  and  instead  offered  an  overly-simplified  stoplight 
model  (2002)  which  unfortunately  gives  no  specific  basis  for  assigning  colors  to  systems, 
and  does  not  provide  for  system-to- system  comparison.  The  Interoperability  Score  (i- 
Score)  (Ford,  et.  al.,  2007a;  2008b)  recognized  that  1)  interoperability  must  be  measured 
in  the  context  of  the  operational  mission,  2)  an  operation  is  implemented  by  systems  of 
many  types,  and  the  interoperability  measurement  must  account  for  them  all,  3)  perfect 
interoperability  is  not  always  desirable  or  possible,  and  4)  it  is  not  the  number  of 
interoperations  that  is  important,  but  the  quality  of  the  interoperations. 
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2.2.3  Extant  Method  Contributions 


Each  method  surveyed  in  sections  2.2.1  and  2.2.2  contributed  towards  the  general 
theory  of  interoperability  measurement  described  in  Chapter  3.  Table  1  lists  their  main 
contributions  (details  in  Appendix  A3)  and  section  2.8  highlights  remaining  gaps  in 
knowledge  which  are  addressed  in  this  dissertation. 


Table  1  Main  contributions  of  extant  interoperability  measurement  methods 


Method 

Main  Contribution 

SoIM 

Interoperability  can  be  measured  in  levels 

QoIM 

Interoperability  can  be  correlated  to  measures  of  effectiveness  via  simulation 

MCISI 

The  distance  between  systems  modeled  as  points  in  space  indicates  their 
interoperability 

LISI 

Systems  possess  interoperability  attributes 

lAM 

Same  as  LISI 

OIM 

Organizations  interoperate,  but  have  different  interoperability  attributes  than 
technical  systems 

Stoplight 

Operations  and  acquisitions  both  have  interoperability  requirements 

LCI 

Operational  interoperability  is  an  extension  of  technical  interoperability 

LCIM 

Conceptual  interoperability  bridges  system  interoperability 

NMI 

Same  as  LISI 

NCW 

Interoperability  occurs  in  the  physical,  information,  cognitive,  and  social  domains; 
lack  of  interoperability  impedes  mission  accomplishment 

SoSI 

System-of-system  research  is  founded  upon  operational,  conceptual,  and 
programmatic  interoperability 

NTI 

Social,  personnel,  and  process  interoperability  are  valid  types  of  non-technical 
interoperability 

OIAM 

There  are  levels  of  ability  of  organizations  to  be  agile  in  their  interoperations 

NID 

Levels  of  interoperability  can  be  described  in  linguistic  terms 

i-Score 

Interoperability  measurements  are  operational  process-specific  and  have  a  maximum 
value 

2.3  Numerical  Taxonomy 

The  science  and  methods  of  numerical  taxonomy  were  introduced  to  the  world  by 
Sokal  and  Sneath  in  1963  in  Principles  of  Numerical  Taxonomy.  Their  later  text 
Numerical  Taxonomy  remains  the  de-facto  handbook  thirty  years  after  its  original 
publication  date.  (Sneath  &  Sokal,  1973)  Sneath  &  Sokal  have  defined  numerical 
taxonomy  as  “the  grouping  by  numerical  methods  of  taxonomic  units  into  taxa  on  the 
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basis  of  their  character  states.”  (1973:4)  While  numerical  taxonomy  has  largely  been 
applied  to  biological,  botanical,  and  genetic  research,  it  has  also  been  used  with  great 
success  in  the  fields  of  ecology,  medicine,  the  social  sciences,  the  earth  sciences,  and  the 
arts  and  humanities.  (Dunn  &  Everitt,  2004)  It  has  only  recently  been  applied  to  systems. 
(Ford,  et.  ah,  2008a)  The  classification  of  life  on  earth  (Maddison,  et.  ah,  2007)  may  be 
the  most  visible  result  of  numerical  taxonomy  and  other  classification  techniques.  The 
methods  and  applications  of  numerical  taxonomy  have  been  documented  in  over  one- 
thousand  articles  and  books.  Numerical  taxonomy  defines  a  science  and  method  used  to 
classify  taxonomic  units  (objects)  according  to  character  states.  (Sneath  &  Sokal,  1973) 
As  such,  the  science  of  numerical  taxonomy  includes  foundational  principles;  a  definition 
of  pertinent  terms  such  as  taxa,  character,  state,  and  cluster;  a  description  of  types  of 
characters  and  states;  methods  of  choosing  characters;  methods  of  estimating  taxonomic 
resemblance;  and  methods  of  identifying  taxa  and  clustering  taxon  into  those  taxa.  (Ibid; 
Dunn  &  Everitt,  2004)  If  a  system  is  also  considered  a  taxonomic  unit  which  can  be 
described  according  to  its  character  states,  then  the  methods  of  numerical  taxonomy  can 
appropriately  be  applied  to  systems  just  as  they  are  applied  to  the  classification  of 
animals,  bacteria,  plants,  language  groups,  minerals,  and  cultures. 

2.4  System  Classification 

The  human  mind  classifies  eveiything  according  to  a  variety  of  factors  including 
morphological,  emotional,  and  functional  among  others.  Expectedly,  all  branches  of 
study  classify.  For  example,  historians  demarcate  events  into  time  periods,  poets  and 
authors  attribute  a  particular  literary  work  to  a  genre,  and  artists  associate  a  work  with  a 
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style.  In  some  disciplines,  however,  the  classification  is  foundationally  important  as  the 
basic  conceptual  and  mathematical  model  of  that  discipline.  For  example,  the  criticality 
of  the  biologists’  empirically  derived  classification  of  organisms  (Linnaeus,  1735)  and 
the  chemists’  quantitative  classification  of  the  elements  (Mendeleev,  1869)  are 
indisputable  to  all  biological  and  chemical  studies  and  analyses.  The  uses  of 
classifications  are  as  numerous  as  the  number  of  types  that  exist  and  include  naming, 
theory  promotion,  lineage  determination,  relationship  demonstration,  and  discovery  of  an 
object’s  nature. 

Several  systems  engineers  have  noted  the  necessity  and  usefulness  of  a  system 
classification.  Maier  and  Magee  &  de  Week  state  that  the  most  important  use  of  a 
classification  lies  in  the  realm  of  system  design  and  analysis.  (1999;  2004)  Shenhar  & 
Bonen  showed  by  example  that  a  system  classification  is  necessary  to  determine  the 
proper  engineering  and  management  style  for  system  development.  (1997)  And  Maier 
pointed  out  that  misclassified  systems  have  problems  not  only  in  design  and 
development,  but  also  in  use.  (1999)  As  Mendeleev’s  classification  predicted  new 
elements  (e.g.,  Germanium)  which  were  later  discovered  (Mendeleev,  1869),  a 
classification  of  systems  can  predict  desirable  future  design  directions  or  new  uses  of 
systems  which,  rather  than  being  discovered,  are  created  by  the  engineer  or  implemented 
by  the  system  operator.  Ford,  et.  ah,  presented  a  mathematically  rigorous  method  of 
classifying  systems  and  postulated  that  if  the  systems  were  classified  according  to  their 
interoperability  characteristics,  that  a  system  interoperability  measurement  could  be  made 
from  the  classification.  A  table  summarizing  generic  system  classifications  (taxonomies) 
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published  in  the  system  science  and  systems  engineering  fields  is  given  in  Table  2  with 
more  detail  provided  in  Appendix  A5. 


Table  2  Historical  summary  of  system  taxonomies  (adapted  from  Ford,  et.  al.,  2QQ8a) 


Year 

Originator 

Basis  of  Taxonomy 

Purpose  of  Taxonomy 

1955 

Von  Bertalanffy 

open/closed 

Launch  General  Systems 

Theory  (GST) 

1956 

Boulding 

complexity 

An  approach  to  GST 

1957 

Goode  & 

Machol 

system  inputs 

Find  solution  to  problems 

1962 

Hall 

none  given 

Partition  subsystems  &  enhance 
meaning  of  “system” 

1985 

Boulding 

world-corresponding 

World  modeling 

1968 

Jordan 

organizing  principles 

Furtherance  of  systems  thinking 

1971 

Ackoff 

system  concepts 

Create  system  concept 
framework 

1981 

Checkland 

origin  of  system 

Group  by  origin 

1990 

Wilson 

none  given 

Refine  definition  of  “system” 

1995, 

Shenhar  & 

technological  uncertainty  & 

Allocate  appropriate  SE  method 

1997 

Bonen 

scope 

to  the  system 

1997 

Martin 

product  type 

Provide  SE  checklist 

1999 

Maier** 

Operational  &  managerial 
independence  of  components 

System-of-system  architecting 

2005 

Gideon  et  al.** 

acquisition  type,  operational 
type,  domain  type 

Aid  in  system-of-system 
understanding 

2005 

Kovacic 

complexity 

Reduce  set  of  systems  into 
meaningful  clusters 

2006 

Blanchard  & 
Fabry  cky 

similarities  &  differences 

Provide  insight  into  wide  range 
of  systems 

2007 

Valdma 

information  classes 

Study  of  non-deterministic 
phenomena 

2008 

Ford,  et.  al. 

similarity  of  system  characters 

Support  system  design 

System  classification  is  important  to  interoperability  measurement  for  the 
following  reasons;  1)  generic  classifications  of  systems  (Table  2)  assist  in  ensuring  that 
all  systems  implementing  an  operational  process  are  identified;  2)  system  classifications 
highlight  characteristics  of  systems,  including  interoperability-related  characteristics;  3) 
quantitative  classifications  of  systems  describe  numerically  the  similarity  between 
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systems;  and  4)  numerical  taxonometric-based  system  classification  provides  a  method 
for  orderly  characterizing  systems. 

2.5  Process  Modeling 

Numerous  process  modeling  methods  and  formats  have  been  published  over  the 
past  several  decades.  A  detailed  survey  was  published  by  Knutilla,  et.  al.  in  1998  which 
identified  common  attributes  of  all  process  modeling  methods  and  formats,  however 
several  important  candidate  operational  process  models  and  formats  were  not  included — 
the  Process  Specification  Language  (PSL)  (NIST,  2007),  the  flow  chart  (IBM,  1969),  the 
SysML  activity  diagram  (OMG,  2007),  and  the  DoD  Architecture  Framework  (DoDAF) 
Activity  &  System  Diagrams  (OV-5,  OV-6a,  b,  c,  SV-5b)  (DoD  2007a,  b,  c).  A 
complete  list  is  given  in  Appendix  A6.  A  process  modeling  method  or  format  used  to 
represent  an  operational  process  in  support  of  an  interoperability  measurement  must  I) 
identify  and  describe  the  operational  tasks  (activities  and  decisions),  2)  describe  the  order 
and  decision  logic  associated  with  the  task  flow,  and  3)  identify  and  associate  systems  to 
tasks.  Mapping  process  modeling  attributes  from  Knutilla,  et.  al.  to  these  requirements 
results  in  the  following. 

•  Req.  #1 :  Identify  and  describe  the  tasks 

o  Simple  Task  Representation  and  Characteristics 
o  Complex  Task  Representation  and  Parameters 

•  Req.  #2:  Describe  the  order  &  decision  logic  associated  with  the  task  flow 

o  Simple  Sequences 
o  Simple  Precedence 


20 


o  Alternative  Task 
o  Complex  Sequences 
o  Concurrent  Tasks 
o  Conditional  Tasks 
o  Iterative  Loops 
o  Parallel  Tasks 
o  Serial  Tasks 
o  Complex  Precedence 

o  Synchronization  of  Multiple,  Parallel  Task  Sequences 
•  Req.  #3:  Identify  &  associate  systems  to  tasks 
o  Resource 

o  Resource  Requirements  for  a  Task 
o  Simple  Resource  Capability/Characteristics 
o  Resource  Allocation/Deallocation  for  One  or  Many  Tasks 
The  following  process  modeling  methods  possess  most  of  the  aforementioned 
attributes  and  are  considered  as  candidate  operational  process  model  formats:  ACT 
Formalism,  I-N-OVA  Constraint  Model,  0-Plan  Task  Formalism,  Virtual  Process 
Modeling  Language  (VPML),  Process  Specification  Language  (PSL),  and  SysML. 
Although  other  methods  could  have  been  chosen,  for  this  dissertation,  the  SysML 
Activity  Diagram  is  the  operational  process  modeling  format  of  choice  because  1)  it  is  an 
emerging  systems  engineering  tool,  2)  it  is  derived  from  and  similar  to  the  ubiquitous 
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Unified  Modeling  Language  (UML)  diagrams,  and  3)  several  popular  software  packages 
available  at  the  Air  Force  Institute  of  Technology  support  it. 

2.6  Operational  Effectiveness  Measurement 

There  has  been  much  published  on  measures  of  effectiveness,  to  include  a  recent 
doctoral  dissertation  (Bullock,  2006),  master’s  theses  (James,  1996;  Bell,  2005),  research 
reports  (Doyle,  et.  ah,  1997;  Gaedecke,  2006),  technical  reports  (including  a  survey 
paper)  (Nelson,  et.  ah,  1996;  Campbell,  2004),  refereed  journal  articles  (Sproles,  2000; 
Sproles,  2001;  Murray,  2001;  Sproles,  2002;  Finkelstein  &  Morawski,  2003;  Finkelstein, 
2003;  Bullock  &  Deckro,  2006),  conference  papers  (Sarle,  1995;  Green,  2001;  Smith  & 
Clark,  2004),  and  workshop  reports  (Sweet,  et.  ah,  1985;  Green  &  Johnson,  2002).  Also 
published  have  been  many  textbooks  (of  which  only  two  are  referenced  here)  (Keeney, 
1992;  Geisler,  2000)  and  several  Department  of  Defense  documents  (Bomman,  1993; 
Stenbit,  et.  ah,  2002a;  USJFCOM,  2005;  DAU,  2006;  DAU,  2006a;  JP  3-0,  2006;  JP  5-0, 
2006;  JP  1,  2007).  Measures  have  many  names,  including  metrics,  measures  of  merit, 
figures  of  merit,  measures  of  effectiveness,  and  measures  of  performance  among  others. 
(Stenbit,  et.  ah,  2002a)  Many  researchers  have  acknowledged  confusion  with  regards  to 
terminology.  (Bell,  2005;  Bullock,  2006;  Green  &  Johnson,  2002;  Green,  2001;  Stenbit, 
et.  ah,  2002;  Nelson,  et.  ah,  1996;  Smith  &  Clark,  2004;  Sproles,  2000;  Sproles,  2001) 
The  term  measure  of  operational  effectiveness  (MoOE)  is  used  in  this  dissertation  and  is 
appropriately  chosen  because  it  1)  reflects  the  importance  of  capturing  the  effectiveness 
of  an  operation  to  the  end  of  describing  how  changes  in  interoperability  affect  operational 
effectiveness;  2)  synchronizes  with  Department  of  Defense  publications  which  define  an 
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MOE  as  a  measure  of  an  effect  (JP  3-0,  2006;  JP  5-0,  2006);  and  3)  fits  properly  within 
the  measurement  hierarchy  (Stenbit,  et.  ah,  2002)  which  has  been  widely  accepted  by 
MOE  researchers.  Appendix  A6  includes  additional  analysis  on  the  hierarchy  of 
measures,  operational  effectiveness  assessment  in  joint  operations,  and  MOE 
characteristics,  types,  and  domains. 

2.7  Architecture 

An  architecture  is  a  depiction  (written  or  drawn)  of  “the  structure  of  components, 
their  relationships,  and  the  principles  and  guidelines  governing  their  design  and  evolution 
over  time.”  (DoD,  2007a)  Curts  &  Campbell  stated,  “without  a  consolidated, 
coordinated,  and  organized  architecture  there  is  little  chance  of  ever  attaining  that  elusive 
goal  of  total  interoperability.”  (1999:1)  The  architecture  description  is  a  possible  source 
and  repository  for  that  which  is  required  to  make  an  interoperability  measurement  as  well 
as  for  decisions  based  upon  the  measurement.  Many  frameworks  exist  which  provide 
guidelines  for  creating  such  an  architecture  description.  The  first  of  these  was  the 
Zachman  Framework  (Zachman,  1987)  followed  by  numerous  others  including  the 
Federal  Enterprise  Architecture  Framework  (FEAF)  (Federal  CIO,  1999),  the  Treasury 
Enterprise  Architecture  Framework  (TEAF)  (Department  of  the  Treasury,  2000),  the 
Open  Group  Architecture  Framework  (TOGAF)  (The  Open  Group,  2003),  and  the  latest 
version  of  the  Department  of  Defense  Architecture  Framework  (DoDAF)  (DoD,  2007a, 
b,  c).  As  can  be  inferred  from  some  of  their  titles  (the  Zachman  framework  and  TOGAF 
excepted),  each  of  these  frameworks  was  developed  for  a  specific  government  agency  or 
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application.  DoDAF’s  relationship  to  interoperability  is  specifically  addressed  in  this 
research. 


Volume  II,  version  1.5  of  DoDAF  mentions  many  uses  of  DoDAF  architecture 
descriptions — one  of  which  is  system  interoperability  analysis.  (DoD,  2007b:2-5,  fig  2-2) 
Specifically,  the  architecture  products  listed  in  Table  3  are  named  as  being  highly, 
partially,  or  non-applicable  to  system  interoperability  uses.  While  some  of  the  elements 
required  for  system  interoperability  measurement  are  stored  cleanly  within  the  DoDAF 
products  in  the  table,  section  3.7  shows  that  some  are  absent,  or  stored  but  not  easily 
extracted.  For  example,  the  interoperability  measurement  method  of  Chapter  3 
accommodates  a  broad  definition  of  the  word  “system”  whereas  DoDAF  defines  a  strong 
separation  between  operational  nodes  and  organizations  and  system  nodes  and  systems. 
This  hampers  its  ability  to  act  cleanly  as  a  source/repository  for  interoperability 
measurement  key  elements. 

Table  3  DoDAF  claimed  product  applicability  to  system  interoperability  (DoD,  2007b) 


Level 

DoDAF  Product 

Highly  applicable 

AV-1,2 

OV-1,2,3,5,  6 
SV-1,4,  6,8 

TV-1 

Partially  applicable 

OV-4,  7 

SV-2,  7,  11 

TV-2 

Not  applicable 

SV-3,  5,  9,  10 

2.8  Gaps  in  Current  Research 

The  following  selected  key  concepts,  which  are  critical  to  a  unified  method  of 
interoperability  measurement,  have  not  been  addressed  by  other  researchers  and  represent 
gaps  in  knowledge.  All  are  addressed  in  Chapter  3.  The  list  is  not  comprehensive,  but  is 
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representative  in  showing  the  magnitude  of  the  eurrent  gap  in  interoperability 
measurement  understanding. 

What  entities  should  have  their  interoperability  measured? 

How  are  systems  modeled  in  support  of  an  interoperability  measurement? 

How  does  a  measure  of  system  similarity  relate  to  interoperability? 

Should  the  similarity  measure  be  allowed  to  be  a  distanee  measure? 

What  does  a  eomplete  framework  of  system  interoperability  attributes  contain? 
What  method  can  be  used  to  identify  interoperability  attributes? 

How  can  the  interoperability  of  two  heterogeneous  systems  be  measured? 

How  are  multiple  interoperability  types  accommodated  in  the  measurement? 

Can  one  measurement  accommodate  different  systems  and  interoperability  types? 
Can  the  interoperability  of  confrontational  (i.e.,  opposing  systems)  be  measured? 
What  is  the  difference  between  collaborative  and  confrontational  interoperability? 
How  do  interoperability  and  operational  effectiveness  relate? 

Is  an  interoperability  measurement  specific  to  an  operational  process? 

Where  should  interoperability  data  be  stored  and  obtained? 

How  interoperable  must  systems  be? 

How  can  an  interoperability  measurement  be  used? 

2.9  Conclusion 

Analysis  of  the  aforementioned  publications  indicates  that  the  following  concepts 
are  important  for  interoperability  measurement:  1)  interoperability  of  sets  of  systems 
(vice  single  systems)  should  be  measured;  2)  the  set  of  systems  should  be  determined  by 
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an  operational  process;  3)  a  framework  of  interoperability  attributes  should  be  used  to 
help  identify  system  interoperability  characters;  4)  systems  should  be  modeled  as  a  set  of 
system  interoperability  character  states;  5)  system  character  states  should  be  identified 
numerically  or  coded  numerically;  6)  a  similarity  function  gives  the  resemblance  between 
systems  as  pertaining  to  their  system  character  states;  7)  the  similarity  function  is  the 
interoperability  measurement  if  the  systems  have  been  appropriately  modeled;  8) 
interoperability  is  related  to  measures  of  operational  effectiveness  associated  with  the 
operational  process;  9)  the  interoperability  of  collaborative  and  confrontational  systems 
can  be  measured;  10)  an  interoperability  architecture  can  be  used  to  supply  and  store  the 
data  supporting  an  interoperability  measurement. 
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3.  Method 


Interoperability  will  never  be  an  analytically  useful  field  of  study 
until  it  is  defined  in  a  quantitative  way. — E.  Presson 


3.1  Overview 

This  chapter  describes  a  general  interoperability  measurement  method  (Figure  4) 
which  can  be  summarized  as  follows.  Given  a  purpose  for  making  an  interoperability 
measurement  and  an  associated  operational  process,  a  set  of  systems  implementing  the 
activities  and  decisions  of  that  process  can  be  identified.  Each  system  in  that  set  can  be 
modeled  quantitatively  as  a  sequence  of  states  of  descriptive  features  of  the  system. 

These  system  models,  called  system  instantiations,  can  be  aligned  with  each  other  and 
their  similarity  measured.  A  fundamental  concept  of  the  interoperability  measurement 
method  is  that  if  the  set  of  systems  is  instantiated  strictly  according  to  interoperability 
features,  then  a  measure  of  the  similarity  of  a  pair  of  system  instantiations  is  a  measure  of 
the  associated  pair  of  systems’  interoperability.  Furthermore,  another  fundamental 
concept  states  that  if  the  systems  are  instantiated  with  interoperability  features  pertinent 
to  a  measure  of  operational  effectiveness  associated  with  a  confrontational  operational 
process,  then  the  interoperability  measurement  can  be  related  mathematically  to  the 
measure  of  operational  effectiveness.  Finally,  the  method  can  be  integrated  with  the 
Department  of  Defense  Architecture  Framework  which  is  suitable  for  storing  key 
interoperability  measurement  elements.  Succeeding  sections  address  the  method  in 
detail. 
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Figure  4.  Interoperability  measurement  method 
3.2  Purpose  of  Interoperability  Measurement 

The  number  of  interoperability  types  is  large  (Appendix  A2)  and  the  number  of 
reasons  to  measure  interoperability  is  even  larger.  Some  reasons  important  to  the 
Department  of  Defense,  extraeted  from  selected  joint  publications  and  the  Department  of 
Defense  System  Engineering  Plan  Preparation  Guide  (DoD,  2008),  are  given  in  Table  4. 
Before  an  interoperability  analysis  is  undertaken,  the  purpose  of  the  analysis  must  be 
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stated.  Identification  of  this  purpose  is  critical  in  keeping  the  analysis  focused  and 
properly  scoped. 


Table  4  Some  DoD  reasons  for  measuring  interoperability 


MULTINATIONAL  OPERATIONS  REASONS 

Collaborative  or 
Confrontational 
Interoperability 

Determine  level  of  integration  and  synchronization  of  coalition 
forces  (JP  3-16) 

Collaborative 

Measure  effectiveness  of  coalition  equipment  (JP  3-16) 

Confrontational 

Troubleshoot  coalition  logistics  problems  (JP  3-16) 

Collaborative 

Determine  mission  impact  of  lack  of  common  tactics,  techniques, 
and  procedures  (JP  3-16) 

Collaborative 

Measure  mission  impact  of  language  and  cultural  difference  among 
coalition  forces  (JP  3-16) 

Collaborative 

Measure  results  of  multinational  planning  and  preparation  on 
mission  success  (JP3-16) 

Collaborative 

Predict  impact  of  liaison  officers  on  coalition  force  mission  success 
(JP  6-0) 

Collaborative 

Provide  metrics  to  5 -nation  Combined  Communications  Electronics 
Board  in  support  of  their  pursuit  of  communications-electronics 
interoperability  (JP  6-0) 

Collaborative 

Provide  COCOM  with  interoperability  reqs.  for  theater  security 
cooperation  plan  (JP  6-0) 

Both 

Determine  impact  of  interface  (translation)  used  to  ensure 
interoperability  of  incompatible  communications  systems  (JP  6-0) 

Both 

Determine  predicted  advantage  over  the  enemy 

Confrontational 

Identify  areas  for  improvement  in  coalition  operations 

Both 

JOINT  OPERATIONS  REASONS 

Determine  impact  of  communications  interoperability  on  personnel 
recovery  (JP  3-50) 

Collaborative 

Determine  impact  of  joint  service  training  on  mission  success 

Collaborative 

Specifying  joint  force  interoperability  requirements 

Both 

Measure  success  of  joint  force  exercises 

Confrontational 

Determine  impact  of  insertion  of  new  communications  technology 
(JP  6-0) 

Both 

Assessing  shortfalls/deficiencies  of  communications  on  operational 
effectiveness  (JP  6-0) 

Both 

Determine  ability  to  cooperate  with  OGAs  and  NGAs  (JP  6-0) 

Collaborative 

Facilitate  CIO  responsibility  to  enforce  interoperability  (JP  6-0) 

Collaborative 

Support  COCOMs  in  verifying  operational  interoperability 
procedures  (JP  6-0) 

Both 
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Measure  interoperability  of  the  joint  communications  system  (JP  6- 
0) 

Collaborative 

Determine  ISR  product  distribution  bottlenecks  (JP  6-0) 

Collaborative 

Measure  system-to-system  compatibility  (JP  6-0) 

Both 

Validate  key  interoperability  solutions  prior  to  mission  execution  (JP 
6-0) 

Both 

Correlate  interoperability  with  the  speed  of  commander 
decisionmaking  (JP  3-27) 

Both 

Predict  advantage  over  the  enemy  in  future  conflicts 

Confrontational 

SYSTEMS  ENGINEERING  REASONS 

Determine  compliance  with  program  certification  requirements 
(SEPPG) 

Both 

Specify  program  requirements  in  terms  of  interoperability 
measurements  (SEPPG) 

Both 

Characterize  system  design  requirements  in  the  System  Engineering 
Plan  (SEPPG) 

Both 

Determine  level  of  interoperability  within  and  without  the  System- 
of-System  (SEPPG) 

Both 

Determine  impact  of  system  configuration  changes  (SEPPG) 

Both 

State  interoperability  key  performance  parameter  requirements 
(SEPPG) 

Both 

Ensure  systems  in  development  will  be  interoperable  with  fielded 
systems  (SEPPG) 

Collaborative 

Facilitate  interoperability  testing  (JP  6-0) 

Both 

HOMELAND  DEFENSE  REASONS 

Measure  and  predict  future  interagency  interoperability  (JP  3-27) 

Collaborative 

Establish  requirements  for  interagency  emergency  response 
communications  (JP  3-27) 

Collaborative 

Measure  ability  of  government  agencies  to  share  information  during 
crisis  (JP  3-27) 

Collaborative 

Measure  operational  connection  between  agencies  in  a  dynamic 
environment  (JP  3-27) 

Collaborative 

Determine  possibility  of  information  overload  as  multiple  first 
responders  use  common  equipment  and  procedures 

Collaborative 

Determine  usefulness  of  inserting  commercial  communication 
standards  into  DoD  acquisition  requirements  (JP  3-27) 

Collaborative 

HEALTH  SERVICE  REASONS 

Justify  standardization  of  medical  capabilities  and  material  with 
other  nations  (JP  4-02) 

Collaborative 

Methodically  measure  compliance  with  OPLAN  (JP  4-02) 

Collaborative 

Discover  medical  training,  logistics,  doctrine,  and  other  concerns  (JP 
4-02) 

Collaborative 
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Determine  communication  interoperability  concerns  between 
MEDEVAC  platforms  and  medical  regulating  authority  (JP  4-02) 

Collaborative 

OTHER  REASONS 

Determine  level  of  net-centricity  of  networked  systems 

Collaborative 

Quantify  Joint  Interoperability  Test  Command  operational 
interoperability  test  requirements  (JP  6-0) 

Both 

3.3  Operational  Process 

After  determining  the  purpose  for  an  interoperability  measurement  analysis,  the 
subjects  of  the  interoperability  measurements  must  be  determined  and  a  method  for 
identifying  them  must  be  given.  In  this  dissertation,  the  operational  process  is  used  to 
identify  a  set  of  systems,  both  friendly  (blue)  and  adversarial  (red),  whose  interoperability 
is  to  be  measured  and  to  identify  the  measure  of  operational  effectiveness  to  which  the 
interoperability  measurement  will  be  correlated.  The  operational  process  is  defined  as  a 
set  of  tasks,  logically  sequenced,  which  when  executed  achieve  a  desired  effect. 
Operational  processes  can  be  collaborative,  which  are  operational  processes  implemented 
by  a  set  of  friendly  systems  working  together  to  achieve  a  shared  goal,  or  confrontational, 
which  are  operational  processes  implemented  by  two  sets  of  systems  acting  in  opposition 
to  each  other  (i.e.,  when  one  set  of  systems  experiences  a  level  of  success  in 
implementing  the  operational  process,  the  other  experiences  a  corresponding  level  of 
failure). 

3.3.1  Modeling  Methods 

While  many  practical  operational  processes  are  simplistic  in  structure,  it  is 
possible  to  define  a  process  in  which  all,  some,  or  none  of  the  tasks  occur  concurrently 
and  in  which  timing  and  logic  are  associated  with  the  elements  of  the  process.  For 
example,  the  finish  of  one  task  may  drive  the  start  of  another,  or  the  decision  of  which 
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task  to  execute  next  might  depend  upon  certain  prerequisites  (including  timing)  being 
met.  Not  surprisingly,  many  methods  and  formats  of  task,  activity,  process,  and  thread 
modeling  have  been  devised  over  the  years.  These  methods  and  formats  are  graphical, 
mathematical,  and  linguistic  in  nature  and  all  have  different  fortes.  Of  three  dozen 
candidate  modeling  methods  and  formats  (Appendix  A6),  the  SysML  activity  diagram  is 
appropriate  for  operational  process  modeling  and  is  the  method  of  choice  for  this 
dissertation  because  it  1)  captures  the  operational  activities  and  decisions  in  a  process,  2) 
describes  what  is  transferred  between  activities  and  decisions,  3)  describes  the  decision 
logic  and  timing  (swim-lane  variant)  of  the  process,  4)  accommodates  the  association  of 
systems  to  activities  and  decisions  as  an  object  attribute,  and  5)  is  supported  by  a  growing 
number  of  software  packages.  An  example  SysML  activity  diagram  with  six  activities 
and  one  decision  is  given  in  Figure  5. 


act  AF  Dynamic  Targe-ling  [Activity  Diagram] ^ 


Figure  5.  Sample  SysML  activity  diagram  used  to  model  an  operational  process 
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3.3.2  Measures  of  Operational  Effectiveness 

An  interoperability  measurement  ean  be  correlated  to  operational  effectiveness  for 
both  collaborative  and  confrontational  operational  processes.  For  a  collaborative 
operational  process,  this  correlation  must  be  done  via  discrete  event  or  other  type  of 
simulation  and  is  outside  the  scope  of  this  research.  An  early  attempt  at  doing  this  was 
made  in  1989  by  Mensh,  et.  al.  in  which  the  interoperability  of  Battle  Force  Command, 
Control,  and  Communications  systems  was  measured  and  correlated  to  measures  of 
effectiveness  and  measures  of  performance  written  as  binary  logic  equations.  For  a 
confrontational  operational  process  however,  this  research  provides  sufficient  conditions 
which  allow  the  interoperability  measurement  to  be  related  to  a  measure  of  operational 
effectiveness  (MoOE)  for  the  operational  process.  The  MoOE  should  be  written  at  a 
level  of  abstraction  equivalent  to  that  of  the  operational  process  and  can  be  a  natural, 
constructed,  or  proxy  measure  originating  from  the  physical,  information,  or  cognitive 
domain  (Appendix  A7).  Ideally  the  MoOE  is  relevant,  measurable,  responsive 
(sensitive),  resourced,  understandable,  discriminatory,  quantitative,  realistic,  objective, 
appropriate,  inclusive,  independent,  valid,  and  reliable.  Additionally,  in  order  to  apply 
the  Interoperability  Impact  on  Operational  Effectiveness  axiom  (section  3.6)  the  MoOE 
must  be  written  diametrically.  In  other  words,  the  MoOE  must  be  written  as  a  pair 

O  =  {Og,Og]  which  relates  the  effectiveness  of  the  set  of  friendly  (blue)  systems  to  the 
lack  of  effectiveness  of  the  set  of  adversary  (red)  systems.  For  example,  =  1 

describes  the  relationship  between  the  diametric  MoOE  O  =  {0g,0^}  where  Og  is  the 
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percent  of  red  force  targets  destroyed  and  is  the  percent  of  red  force  targets  protected. 

While  many  MOEs  contained  in  the  Universal  Joint  Task  List  (UJTL)  (CJCSM 
3500. 04D,  2006)  cannot  be  written  as  diametric  MoOEs,  such  as  OP  2.2.1,  M31, 
“Minutes  to  determine  raid  size.”,  many  others  can.  For  example,  numerous  percentage 
MOEs  in  the  UJTL  such  as  SN  3.4,  M7  “percent  of  cruise  missiles  destroyed  before 
impact,”  ST  1.3.5,  M5  “percent  of  enemy  forces  drawn  away  from  main  thrust  after 
demonstration,”  or  OP  1.2.4. 7,  M13  “percent  of  friendly  personnel  recovered  uninjured” 
can  be  all  be  written  diametrically.  Similarly,  MOEs  in  the  UJTL  which  represent  a 
count  such  as  SN  3.4.1,  Ml  1  “number  of  safe  passage  aircraft  engaged,”  can  be 
normalized  and  converted  to  a  percentage  (e.g.,  “percent  of  safe  passage  aircraft 
engaged”)  and  then  written  diametrically.  In  general,  UJTL  MOEs  which  can  be 
converted  to  a  percentage  and  which  correlate  friendly  force  action  to  enemy  force 
reaction  (or  vice  versa)  are  candidate  diametric  MoOEs. 

3.4  Systems 

Among  others,  the  interoperability  of  processes,  enterprises,  organizations, 
coalitions,  concepts,  functions,  objects,  products,  models,  cultures,  doctrines,  forces, 
cities,  public  services,  and  applications  can  be  measured,  however,  systems  are  chosen  as 
the  object  of  the  interoperability  measurement  in  this  research.  The  word  “system”  is 
used  in  a  variety  of  fields  of  study,  but  its  definition  is  fairly  standardized  across  those 
disciplines.  For  example,  the  military  scientist  defines  system  as  “a  functionally, 
physically,  and/or  behaviorally  related  group  of  regularly  interacting  or  interdependent 
elements;  that  group  of  elements  forming  a  unified  whole.”  (JP  1-02,  2008:534)  The 
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system  engineer  defines  it  as  “a  combination  of  interacting  elements  organized  to  achieve 
one  or  more  stated  purposes.”  (Haskins,  2007:1.5)  The  management  scientist  defines  it 
as  “an  entity  which  is  comprised  of  at  least  two  elements  and  a  relation  that  holds 
between  each  of  its  elements  and  at  least  one  other  element  in  the  sef  ’  (Ackoff, 

1971:662).  And  finally,  the  physician  defines  it  as  “a  consistent  and  complex  whole 
made  up  of  correlated  and  semi-independent  parts;  a  complex  of  functionally  related 
anatomic  structures.”  (Pugh,  2000:1775)  There  are  three  common  themes  to  these 
definitions:  1)  systems  are  comprised  of  a  set  of  elements,  2)  those  elements  interact,  and 
3)  the  interacting  set  of  elements  forms  a  whole  and  act  in  concert  to  achieve  the  system’s 
purpose.  Therefore,  in  this  research  a  system  is  defined  as  “an  entity  comprised  of 
related  interacting  elements,  which  act  together  to  achieve  a  purpose.”  This  definition  is 
broad  enough  to  include  a  wide  variety  of  systems  including,  but  not  limited  to,  technical, 
biological,  environmental,  organizational,  conceptual,  physical,  and  philosophical,  among 
others.  Because  the  definition  of  system  used  in  this  research  is  broad,  the  system 
interoperability  measurement  method  presented  is  applicable  to  an  equally  broad  set  of 
entities. 

Other  interoperability  researchers  also  promote  measuring  the  interoperability  of 
systems.  Nine  of  the  fourteen  interoperability  measurement  method  papers  surveyed 
(section  2.2),  directly  advocated  the  measurement  of  system  interoperability.  (LaVean, 
1980;  Mensh,  et.  al.,  1989;  Amanowicz  &  Gajewski;  1996;  DoD  1998;  Leite,  1989; 
Hamilton,  et.  al.,  2002;  NATO,  2003;  Stewart,  et.  al,  2004;  Ford,  et.  al.,  2007a;  2008b) 
The  other  five  recommended  measuring  the  interoperability  of  an  entity  that  could  be 
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modeled  as  a  system  or  as  a  system  enabler.  (Clark  &  Jones,  1999;  Tolk  &  Muguira, 
2003;  Tolk,  2003;  Stewart,  et.  al.,  2004;  Kingston,  et.  al.,  2005;).  Although  six  of  the 
fourteen  methods  surveyed  proposed  measuring  the  interoperability  of  singleton  systems 
(Leite,  1989;  Clark  &  Jones,  1999;  Hamilton,  et.  al.,  2002;  Tolk  &  Muguira,  2003;  Tolk, 
2003;  Kingston,  et.  al.,  2005),  measuring  the  interoperability  of  a  single  system  is 
antithetical  to  the  connotation  of  the  word  interoperability  and  to  the  definition  of  the 
word  as  used  in  this  research.  Measuring  the  interoperability  of  a  set  of  two  or  more 
systems  is  a  better  choice.  But  as  the  set  of  all  systems  is  infinite,  a  means  of  limiting  the 
size  of  the  set  is  needed. 

3.4.1  Constraining  the  Set  of  Systems  with  an  Operational  Process 

Although  many  methods  exist  for  determining  which  systems  should  have  their 
interoperability  measured,  using  the  operational  process  to  determine  the  set  of  systems  is 
appropriate  for  at  least  three  reasons.  First,  systems  perform  different  interoperations  in 
different  scenarios  (i.e.,  they  are  used  differently);  second,  effectiveness  is  often 
measured  at  the  operational  process  level  (i.e.,  measures  of  effectiveness);  and  third, 
operational  processes  can  be  written  at  any  level  of  abstraction  which  enables  system 
definition  at  the  same  level  of  abstraction.  When  using  an  operational  process  to 
constrain  the  set  of  systems,  all  systems  are  identified  which  implement  the  operational 
process.  The  set  of  systems  is  often  diverse  and  can  be  small  or  large. 

3.4.2  System  Identification 

There  are  a  variety  of  methods  which  can  be  used,  within  the  constraints  of  an 
operational  process,  to  identify  the  set  of  systems  -S  =  |5'j,52,. .  whose 
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interoperability  is  to  be  measured.  These  methods  include  architectural  methods  such  as 
Activity  Based  Modeling  (Ring,  et.  al.,  2004)  and  DoD  Architecture  Framework- 
associated  structured  analysis  modeling  (DoD,  2007a,  b,  c),  engineering  methods  such  as 
IDEFO  and  IDEF3,  project  methods  such  as  the  Microsoft  Project  method  (2007),  process 
methods  such  as  SySML  activity /decision  to  resource  (system)  association  (OMG,  2007), 
as  well  as  others.  While  seemingly  simplistic,  it  is  easy  to  neglect  certain  types  of 
systems  (e.g.,  weather  systems)  which  may  not  be  routinely  considered  as  such,  but 
which  might  be  important  in  the  final  interoperability  analysis.  For  this  reason,  any 
system  identification  method  chosen  can  be  complemented  by  the  use  of  a  generic  system 
taxonomy.  These  comprehensive  taxonomies,  while  simplistic  and  general  in  nature,  are 
reminders  of  the  wide  variety  of  systems  that  exist.  Researchers  in  the  fields  of  system 
science  and  systems  engineering  have  published  taxonomies  of  systems  for  at  least  fifty 
years.  A  survey  of  these  taxonomies  is  given  in  appendix  A5. 

3.4.3  System  Characterization 

Once  the  set  of  systems  S  has  been  identified,  those  systems  must  be  modeled. 
Applying  numerical  taxonomic  concepts,  a  system  can  be  modeled  using  a  set  of 
characters  X  =  |xj,X2,. .  which  represent  traits,  attributes,  or  characteristics  which 

describe  the  important  features  of  the  system.  These  system  characters  can  be 
morphological  (e.g.,  size,  shape,  color,  structure,  type  and  method  of  construction, 
material  composition,  or  number  of  components),  physiological  (e.g.,  system  functions  or 
behaviors),  interfacial  (e.g.,  type  and  number  of  system  or  element  interfaces),  ecological 
(e.g.,  system  context,  environment,  or  type  of  fuel  consumed),  and  distributional  (e.g., 
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geographic  location  or  domain)  among  others.  Ideally,  the  set  of  characters  chosen 
should  be  natural  (i.e.,  a  character  not  confounded  with  another)  and  diagnostic  (i.e.,  a 
character  which  distinguishes  one  system,  or  system  type,  from  another).  Additionally, 
the  types  of  characters  chosen  should  be  related  to  the  type  of  interoperability 
measurement  that  is  to  be  undertaken.  For  example,  interoperability  measurement  of 
devices  on  a  network  demands  that  functional  characters  be  emphasized,  whereas  a 
spatial  interoperability  analysis  requires  that  certain  morphological  characters  should  be 
used. 

Extending  a  definition  from  the  phylogeneticists  (Semple  &  Steel,  2003), 
characters  are  functions  which  map  systems  in  S  to  the  states  C  of  their  characters  X 

where  the  set  of  valid  character  states  for  a  set  of  characters  is  C  =  {cj,C2 . .  .,c„}  (see 

definition  below).  Character  states  are  either  qualitative  (discrete)  or  quantitative 
(discrete  or  continuous),  or  a  mixture  of  both  (Sneath  &  Sokal,  1973).  Generally,  the  set 
of  character  states  is  restricted  to  the  binary  numbers  (absence/presence  states)  or  the 
positive  real  numbers,  although  other  states  are  certainly  possible. 

DEFINITION  (System  Characterization):  Given  a  set  of  systems  S ,  then 
X  :S  is  a  function  which  maps  systems  to  a  set  of  character 
states  C  and  X  is  called  the  characterization  of  S . 

3.4.3. 1  Interoperability  Characters 

As  mentioned  previously,  there  are  numerous  types  of  system  characters  that  can 
be  used  to  describe  a  system.  However,  in  order  to  form  a  basis  for  an  interoperability 
measurement,  only  a  special  type  of  system  characters  should  be  used  to  model  a  system; 
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these  are  ealled  system  interoperability  characters,  or  interoperability  characters  for  short. 
This  is  a  foundational  concept  in  interoperability  measurement.  In  their  essence, 
interoperability  characters  describe  what  systems  do  to  each  other.  For  example, 
opposing  forces  attack  each  other,  two  computers  communicate  with  each  other,  and 
two  businesses  trade  with  each  other.  In  these  three  examples,  the  words  attack , 
communicate ,  and  trade  all  imply  a  type  of  interoperation. 

It  is  not  possible  to  list  all  interoperability  characters,  however  it  is  important  to 
note  that  generally  any  type  of  character  is  an  interoperability  character  in  specific 
circumstances.  For  example,  although  physiological  and  interfacial  characters  are  clearly 
interoperability  characters,  the  morphological  character  shape  becomes  an 
interoperability  character  when  the  docking  interoperation  of  the  space  shuttle  and  the 
international  space  station  is  measured.  Similarly,  the  distributional  character  of  domain 
is  an  interoperability  character  when  considering  the  environment  in  which  systems  are 
used.  For  example,  a  Navy  destroyer  is  interoperable  with  the  ocean  but  not  the  land,  yet 
the  Marine  Corps’  Expeditionary  Fighting  Vehicle  (EFV)  is  interoperable  with  both. 

An  interoperability  character  represents  a  pair  of  actions,  such  as  “provide”  and 
“accept,”  which  constitute  an  interoperation.  These  pairs  describe  how  systems  provide 
and  accept  matter,  energy,  or  information  from  each  other.  A  selection  of  interoperability 
characters  associated  with  pairs  of  actions  and  type  of  intended  interoperability 
measurement  is  given  in  Table  5.  While  not  exhaustive,  it  gives  a  sample  of  the  many 
types  of  interoperability  actions  performed  by  systems.  An  example  framework  of  some 
interoperability  characters  arranged  by  system  type  is  given  in  Table  34  in  Appendix  A8. 
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Table  5  A  selection  of  interoperability  pairs  and  associated  characters 


Interoperability  Pairs 

Interoperability  Type 

Character 

Provide  <:^>  Accept 

General 

Interoperate 

Transmit  <:^>  Receive 

Communication 

Communicate 

Attack  Attacked 

Confrontational 

Attack 

Impact  Impacted 

Confrontational 

Impact 

Detect  Detected 

Technological 

Detect 

Publish  Subscribe 

Net-Centric 

Service 

Occupy  <:^>  Accommodate 

Spatial 

Accommodate 

Serve  Be  Served 

Human 

Service 

Give  <:^>  Take 

Human 

Share 

Buy  Sell 

Business 

Trade 

Pay  Get  Paid 

Financial 

Transact 

Output  <:^>  Input 

Traditional  System 

Outputinput 

Lead  Follow 

Organizational 

Dance 

Order  Obey 

Human,  Organizational 

Command 

Produce  Consume 

Business,  Human 

Economy 

Transport  Transported  by 

Business 

Transport 

3.4.3.2  Interoperability  Character  Identification 

Interoperability  characters  can  be  extracted  in  a  methodical  fashion  from 
sentences  which  describe  what  systems  do.  These  sentences  may  originate  in 
requirements,  architecture,  and  a  host  of  other  acquisition,  capabilities,  and  operations 
documents.  A  guide  for  relating  the  parts  of  speech  commonly  found  in  sentences  to 
system  interoperability  measurement  is  given  in  Table  6.  Note  that  the  table  does  not 
give  definite  relationships.  For  example,  the  table  does  not  insist  that  all  nouns  must  be 
systems,  as  some  nouns  are  simply  objects  of  verbs. 
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Table  6  Guide  for  relating  parts  of  speech  to  interoperability  modeling 


Part  of  Speech 

Relationship 

Noun 

System  or  refinement  of  an  interoperability  character 

Verb 

Interoperability  character 

Pronoun 

System 

Adverb 

Refinement  of  an  interoperability  character 

Adjective 

Refinement  of  a  system 

Preposition 

Refinement  of  an  interoperability  character 

Conjunction 

Not  applicable 

Interjection 

Not  applicable 

Article 

Not  applicable 

To  illustrate,  the  following  sentence  is  given.  “The  long  train  expeditiously 
transports  raw  material  down  the  tracks  to  the  factory.”  Two  of  the  four  nouns  (train, 
factory)  in  the  sentence  can  be  considered  as  systems.  The  two  adjectives  refine  the 
names  of  the  systems  (long  train,  raw  material),  or  imply  the  existence  of  other  systems 
(e.g.,  short  trains  or  refined  material).  The  single  verb  (transports)  in  the  sentence  implies 
interoperation  between  system  pairs  and  is  an  interoperability  character.  The  adverb 
(expeditiously)  refines  the  interoperability  character  (transports)  implying  that  there  may 
be  different  types,  or  levels,  of  hauling  (e.g.,  sluggishly  transports).  And  the  remaining 
two  nouns  (material,  tracks)  also  refine  the  interoperability  character  by  describing  what 
is  transported  and  by  which  method  it  is  transported.  Similarly,  the  prepositional  phrase 
(“down  the  tracks”)  also  refines  the  interoperability  character  (transports).  The  last 
prepositional  phrase  (“to  the  factory”)  hints  at  the  fact  that  the  train  and  the  factory 
interoperate  (e.g.,  by  providing  and  accepting  raw  material).  From  this  analysis,  it  can  be 
seen  that  a  hierarchical  description  of  interoperability  characters  can  be  made.  If  the 
interoperability  pair  (transports  <:^>  transported  by)  is  represented  by  the  character 
Transport ,  then  refining  interoperability  characters  can  be  Transport. Material , 
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Transport. Material. onTracks  ,  and  Transport. Material. onTracks. Expeditiously  .  If  the 


character  states  are  assumed  to  be  binary  then  the  beginnings  of  a  system  interoperability 
model  can  be  given  in  (1).  If  a  new  system  is  added  to  the  model,  for  example,  a  truck , 
then  it  can  be  seen  that  the  truck  can  also  Transport  raw  material,  but  on  the  road  rather 
than  on  the  tracks  (i.e.,  Transport. Material. onRoad  ). 


S  =  [train, material, tracks,  factory^ 


X  =  \ 


Transport, 

Transport. Material, 

Transport. Material. on  Tracks, 

Transport. Material. onTracks. Expeditiously 


C  =  {0,1} 


(1) 


3.4.3.3  Interoperability  Character  Directionality 

Interoperability  characters  are  inherently  directional.  As  previously  mentioned, 
interoperability  involves  a  pair  of  systems  doing  something  to  each  other.  For  example, 
given  two  systems  Xj  and  ,  assume  that  Xj  attacks  S2.  From  both  systems’ 


perspectives,  Xj  is  initiating  the  attack  (i.e.,  transmitting)  while  is  absorbing  the  attack 
(i.e.,  receiving).  Thus  the  interoperation  (e.g.,  X  =  [Attack^ )  between  Xj  and  S2  is 

directional  from  Xj  to  ^'2 ,  but  not  vice  versa.  The  directionality  of  system  interoperation 
can  modeled  four  different  ways  (Figure  6). 
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Figure  6.  Directional  interoperability  possibilities 
Thus,  when  an  interoperation  occurs  between  two  systems,  the  direction  of  the 
interoperation  must  be  captured.  This  directionality  can  be  annotated  by  a  (T)  for 
transmit  or  a  (R)  for  receive  appended  to  each  interoperability  character  code.  For 

example,  X  =  \^Attack{T),Attack{R)^  is  the  complete  characterization  for  the  example  in 

the  previous  paragraph.  Generalizing,  X  =  .  Although  the  size  of  X  doubles 

in  order  to  accommodate  the  directionality  of  interoperability  characters,  it  is  important  to 
keep  track  of  one-way  interoperations  between  systems.  Both  collaborative  and 
confrontational  interoperations  can  be  directional.  If  every  interoperability  character  in 
X  is  bi-directional,  then  the  (T)  and  (R)  suffixes  are  not  needed. 

3.4.4  System  Instantiation 

Once  systems,  their  interoperability  characters,  and  the  states  of  those  characters 
have  been  identified,  then  a  specific  system  can  be  modeled,  or  instantiated,  as  a 
sequence  (Bullock,  2006;  Amanowicz  &  Gajewski,  1996)  of  states  of  system  characters. 
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DEFINITION  (System  Instantiation):  Given  a  specific  s  gS  and  a  set 
x^X  of  system  characters  descriptive  of  s ,  then  a  =  x(^s^  is  a 

sequence  of  system  character  states,  called  the  instantiation  of  s , 
which  models  s . 

Once  all  sgS  have  been  instantiated,  the  system  instantiations  must  be  aligned 
with  each  other  in  order  to  support  meaningful  system  comparisons  and  other 
mathematical  operations.  Unless  otherwise  stated,  hereafter  the  term  system  instantiation 
implies  an  aligned  system  instantiation. 

DEFINITION  (Instantiation  Alignment):  Given  a  set  x'  c  X  of  system 
characters  descriptive  of  s'  and  a  set  x"  of  system  characters 

descriptive  of  s" ,  and  X  =  then  two  system  instantiations 

cr'  and  cr"  are  aligned  if  (j'  =  X(^s'^  and  cr''  =  X(5'').  The  aligned 
instantiation  of  S  is  given  by  S  =  X(5') . 

In  order  to  illustrate  these  concepts,  an  example  is  given.  Let  5  =  {5j,X2,X3}  be  a 
set  of  systems  of  interest,  let  X  =  |xj,X2,X3,X4}  be  a  set  of  bi-directional  interoperability 
characters  used  to  characterize  S ,  and  let  all  character  states  be  absence/presence  states 
(i.e.,  C  =  {0,l| ).  Define  individual,  unaligned,  system  instantiations  as  in  (2),  then  an 
aligned  instantiation  of  S  is  given  by  (3). 
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(7i  =  {Xi  (^1 ) ,  X2  (^1  ) ,  Xj  (^1  ) ,  X4  (^1  )}  =  {1, 1, 0, 1} 

={^1  (^2  )}  =  {!}  (2) 

0-3  ={X2  (^3  ),X4(^3  )}  =  {!, 0} 

'1  1  0  r 

2  =  X(5)  =  {{1, 1,0,1}, {1,0, 0,0}, {0,1, 0,0}}=  1  0  0  0  (3) 

[o  1  0  0 

3.4.5  Performance  Enhanced  Instantiation 

A  system  instantiation  2  in  which  interoperability  characters  are  assigned  binary 
character  states  is  an  underlying  interoperability  model  upon  which  a  performance- 
enhanced  instantiation  is  based.  The  performance-enhanced  instantiation  can  be  used  to 
facilitate  data  rate,  cost,  efficiency,  or  throughput  analysis,  among  others.  For  example, 
given  a  set  of  systems  (4)  and  a  set  of  bi-directional  interoperability  characters  (5)  with 
character  states  (6),  then  S  can  be  instantiated  as  (7)  if  it  is  assumed  that  the  Laptop  has 
USB,  Wi-Fi,  and  Serial  communication  capability  and  that  the  PDA  possesses  the  same 
plus  GSM,  IR,  and  Bluetooth  functionality. 

S  =  [Laptop,  pda}  (4) 

X  =  [  CommUSB,  CommWiFi,  CommSerial,  Comm.GSM,  Comm.Bluetooth}  (5) 

C  =  {0,1}  (6) 


USB 

WiFi  Serial 

GSM 

IR 

Bluetooth 

2  = 

Laptop  1 

1  1 

0 

0 

0 

PDA  1 

1  1 

1 

1 

1 
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If  a  data  rate  focused  interoperability  analysis  is  desired,  a  performance-enhanced 


system  instantiation  2^  can  be  defined  (8)  which  models,  for  example,  the  peak  data  rates 
(in  Mbits/sec)  for  each  type  of  interoperation  characterized  by  X  . 


USB 

WiFi  Serial 

IR 

GSM 

Bluetooth 

2p  = 

Laptop 

480 

54 

0.02 

0 

0 

0 

PDA 

480 

11 

0.250 

0.271 

1.152 

2 

This  performance-enhanced  system  instantiation  assumes  that  the  Laptop  and 
PDA  adhere  to  the  standards  given  in  Table  7. 


Table  7  System  implementation  standards 

Laptop  pda 


Comm.USB 

USB  2.0 

USB  2.0 

Comm.WiFi 

802.11g 

802.11b 

Comm.Serial 

RS-232  (strict) 

RS-232  (relaxed) 

Comm.GSM 

N/A 

GSM 

CommJR 

N/A 

MIR  IrDA 

Comm.Bluetooth 

N/A 

Bluetooth  2.0 

3.5  Interoperability  Measurement 

Metrology,  or  the  science  of  measurement,  defines  measurement  as  “the  objective 
representation  of  our  empirical  knowledge  of  the  world  by  numbers,”  or  in  other  words, 
“the  assignment  of  numbers  to  properties  or  events  in  the  real  world  by  means  of  an 
objective  empirical  operation,  in  such  a  way  as  to  describe  them.”  (Finkelstein  & 
Leaning,  1984:25-26)  If  interoperability  is  considered  as  a  property  of  a  set  of  systems, 
then  an  operation,  called  a  system  interoperability  measurement,  can  be  defined  which 
objectively  and  empirically  assigns  a  number  to  systems  interoperability.  This  operation, 
its  derivation,  and  its  varieties  are  defined  in  succeeding  sections. 
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3.5.1  System  Similarity 

Similarity  measures  have  been  well-studied.  (Guan,  et.  ah,  2008)  In  its  essence, 
similarity  reflects  the  degree  of  resemblance  of  two  or  more  objects.  In  this  research,  the 
similarity  of  systems  is  measured  by  using  a  function  which  takes  aligned  system 
instantiations  as  its  arguments,  which  rewards  for  shared  interoperability  characters,  and 
which  (optionally)  penalizes  for  unshared  interoperability  characters. 

DEFINITION  (System  Similarity):  Given  aligned  sequences  a',  a" 
instantiating  two  systems  s',s" ,  then  the  similarity  between  s',s"  is 

given  by  Sim[a',(T''^  where  Sim[a',(T"^  is  a  similarity  function. 

There  are  numerous  types  of  candidate  similarity  functions  to  choose  from. 

Sneath  &  Sokal  categorized  similarity  functions  as  distance,  association,  correlation,  and 
probabilistic  measures.  (1973)  Guan,  et.  ah,  offered  a  new  classification,  describing 
similarity  measures  as  geometric,  feature  contrast,  alignment-based,  and  transformational 
measures.  (2008)  Each  type  is  briefly  addressed  below,  followed  by  definitions  of  two 
similarity  functions  especially  appropriate  for  interoperability  measurement. 

Distance  (geometric)  functions  measure  how  far  apart  objects  reside  in  character 
space.  In  other  words,  the  geometric  function  is  a  measure  of  dissimilarity  vice  similarity 
which  is  a  measure  of  how  close  objects  reside  to  each  other  in  character  space.  (Sneath 
&  Sokal,  1973)  A  geometric  dissimilarity  function  can  be  converted  to  a  similarity 
function  by  first  normalizing  the  function  by  maximum  character  state  value  and  by 

number  of  characters  n  so  that  its  range  is  [0,l] ,  then  subtracting  it  from  1.  Thus,  perfect 
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similarity  implies  5'/m(cr',cr")  =  1  and  no  similarity  implies  =  0 .  For 

example,  given  the  Minkowski  distance  function  (9),  the  corresponding  Minkowski 
similarity  function  can  be  written  as  (10).  ,  as  defined  later  in  this  section,  can  be 

classified  as  a  weighted  geometric  similarity  measure,  and  is  used  to  measure  the 
similarity  of  systems  instantiated  with  positive  real-valued  character  states. 

(9) 


Minkowski  Distance  =  MD  =  (cr',cr") 


Minkowski  Similarity  =  MS  =  1 


Z 


V  ^max 


(10) 


Association  similarity  measures  are  related  to  distance/geometric  similarity 
measures  but  are  more  appropriately  described  as  feature,  or  character,  contrast  measures. 
They  are  especially  appropriate  for  measuring  the  similarity  of  objects  described  in 
character  space  with  absence/presence  character  states.  (Batagelj  &  Bren,  1995;  Sneath  & 
Sokal,  1973;  Baulieu,  1989)  The  general  form  of  a  character  contrast  measure  is  given  in 
(11)  where  6,a,p  are  weights,  /  is  a  function,  cr'ncr"  represents  the  features  that 
cr',  cr"  have  in  common,  cr'  -  a"  represents  the  features  that  cr'  possesses  that  cr"  does 
not,  and  cr"  -  cr'  represents  the  opposite.  (Guan,  et.  ah,  2008)  Sim^^^ ,  as  defined  later  in 

this  section,  can  be  considered  a  character  contrast  similarity  measure  and  is  used  to 
measure  the  resemblance  of  systems  instantiated  with  binary  (absence/presence) 
character  states.  Other  examples  of  character  contrast  measures  include  the  Jaccard 
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coefficient,  the  Czekanowski  metric,  and  the  Simple  Matching  coefficient,  among  many 
others;  Batagelj  &  Bren  compare  over  twenty  character  contrast  measures  (1995). 

5'/m(cr',cr'')  =  6*/(cr'ncr'')-a/(cr'-cr'')-y^/(cr"-cr')  (11) 

Correlation  coefficients  associate  the  similarity  of  objects  modeled  in  character 
space  with  a  function  of  the  angle  between  the  object  vectors.  (Sneath  &  Sokal,  1973) 

An  example  of  a  correlation  coefficient  is  the  cosine  function.  Probabilistic  similarity 
measures  account  for  the  distribution  of  character  states  for  a  particular  character  (Ibid) 
and  alignment-based  similarity  measures  give  greater  weight  to  common  features  of 
objects  which  are  related  or  belong  to  the  same  sub-object.  (Guan,  et.  ah,  2008)  Finally, 
transformational  similarity  measures  associate  similarity  with  the  number  of  operations 
required  to  transform  one  object  so  as  to  become  identical  to  another.  (Hahn,  et.  ah,  2003) 

An  interoperability  function  is  a  similarity  function  which  meets  certain  criteria 
given  in  the  definition  below. 

DEFINITION  (Interoperability  Function):  An  interoperability  function  I 
is  a  similarity  function  which  I)  takes  a  pair  of  system  instantiations  as 
its  arguments,  2)  has  a  range  of  [0,l],  3)  rewards  for  shared 

characters  and  optionally  penalizes  for  unshared  characters,  and  4) 
gives  a  greater  reward  to  system  pairs  whose  shared  characters  ’  states 
have  a  “better”  value. 

Although  many  interoperability  functions  might  be  appropriate  for 
interoperability  measurement,  two  fundamental  interoperability  functions  previously 
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alluded  to,  Sinig^^  and  ,  are  defined  below.  is  a  character  contrast 

similarity  function  used  to  measure  the  similarity  of  system  instantiations  with  binary¬ 
valued  (absence/presence)  character  states.  It  can  be  modified  in  many  ways  which  could 
be  useful  for  certain  applications.  For  example,  Sinig^^  does  not  penalize  for  unshared 

interoperability  characters,  although  a  penalty  term  could  be  easily  added.  A  penalty 
might  be  desired  when  the  lack  of  a  certain  interoperability  character  is  a  severe 
detriment  to  the  effectiveness  of  the  operational  process.  adheres  to  the  standard 

character  contrast  measure  form  given  in  (1 1)  with  0  =  y„,  a  =  f3  =  Q ,  and 

/  =  (c7'Aa-). 

DEFINITION  ( Sim^.^):  Given  a  pair  of  systems  s',s"  instantiated  with 

n 

(7',(7"e{0,l}",  then  I  =  =(X)2(  cr'(z)  is  an 

i=\ 

interoperability  function  which  gives  a  normalized  measure  of  the 
similarity  of  systems  instantiated  with  binary-valued  character  states 
where  a  is  the  boolean  AND  operator. 

Sintg^^i ,  is  a  normalized  geometric  measure  appropriate  for  measuring  the 
similarity  of  system  instantiations  with  real-valued  character  states.  It  assumes  that  the 
range  of  all  characters’  states  are  the  same  [0,Cj^] ,  either  inherently  or  by  mapping.  The 

core  of  Sim^^^i  is  the  Minkowski  similarity  function  (10)  modified  to  reward  strictly  for 
shared  characters  by  inserting  the  b^  parameter  defined  in  (13).  The  modified  Minkowski 
similarity  function  is  given  in  (12). 


50 


Modified  Minkowski  Similarity  =  MMS 


(12) 


(,,=jO  <-'{'■)  =  0  0'' <-'{'■)  =  0  (13) 

1  else 


Although  it  would  be  desirable  to  just  use  the  Modified  Minkowski  Similarity 
function  as  Sim^^^, ,  this  is  not  possible  because  it  violates  the  fourth  criteria  of  an 

interoperability  function  by  not  giving  a  greater  reward  to  system  pairs  whose  shared 
characters’  states  have  a  better  value.  For  example,  consider  four  system  instantiations 

(Tj  =  [1] ,  (T^  =  [2] ,  (Tj  =  [3] ,  <7^  =  [4]  and  assume  r  =  2  and  =  4 .  Each  system 
instantiation  possesses  one  (the  same)  interoperability  character,  i.e.,  n  =  l,  but  each 
exhibits  a  different  state  of  that  character.  As  expected,  MMS  (cTj  ,  cr2 )  =  0.75 ,  but 

MMS{(J^,(t^^  =  0.15  as  well.  In  other  words,  cTj  resembles  <T2  just  as  much  as  cTj 
resembles  ,  but  intuitively,  the  similarity  of  should  be  higher  because  those  two 

systems  possess  larger  character  state  values  than  cTj, (T2 .  Hence,  a  weighting  must  be 

applied  to  the  similarity  measurement  to  correct  this  deficiency.  Although  numerous 
weighting  schemes  could  be  chosen,  the  average  character  state  value  (14)  provides  a 
simple,  yet  appropriate,  weighting  scheme  and  is  chosen  to  finalize  the  definition  of 
Sim^^i .  Sim^^^i  has  the  capability  of  yielding  very  precise  similarity  measures  of  system 

instantiations  limited  only  by  the  number  of  characters  and  the  precision  of  those 
characters’  states. 
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Average  Character  State  Value  =  w  =  — - — -  (14) 

2nc 

max 

DEFINITION  ( Sintj^^i):  Given  a  pair  of  systems  s',s”  instantiated  as 

a',  a"  e  M"  '^[0»<^niax]  >  ^  ~  ^^^Reai  ~  W'MMS ,  written  out 

completely  in  (15),  is  an  interoperability  function  which  gives  a 
weighted,  normalized  measure  of  the  similarity  of  systems  instantiated 
with  real-valued  character  states  where  w  is  the  average  character 
state  value  of  a  pair  of  system  instantiations,  MMS  is  the  Modified 
Minkowski  Similarity  function,  n  is  the  number  of  characters  used  to 
instantiate  a', a",  c^^  is  the  maximum  character  state  value,  and  r  is 


3.5.2  System  Interoperability 

Applying  the  concept  of  similarity  to  interoperability  measurement,  a 
foundational  axiomatic  relationship  between  similarity  of  systems  and  interoperability  of 
systems  can  be  stated. 

AXIOM  (System  Similarity  and  Interoperability):  If  a  pair  of  systems  is 

instantiated  only  with  system  interoperability  characters,  then  the  measure  of 
their  similarity  is  also  a  measure  of  their  interoperability. 
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The  System  Similarity  and  Interoperability  axiom  ean  be  used  to  formally  define 
interoperability  measurement. 

DEFINITION  (Interoperability  Measurement) :  Given  two  systems 

s’,s’'  e  S  instantiated  with  bi-directional  characters  as  a',  a"  and  an 

interoperability  function  I ,  then  m  =  I  (cr',cr'')  is  a  measure  of  the 
interoperability  of  s'  and  s"  where  m  =  0  — >  a’,<j"  are  non- 
interoperable  and  w  =  1  — >  cr',  cr"  are  perfectly  interoperable. 

M  =  ^my  j ,  i,  7  ^  1‘S'I  is  a  matrix  of  interoperability  measurements  for 
all  system  pairs  in  S . 

If  the  interoperability  eharaeters  used  to  instantiate  systems  are  directional  in 
nature  (i.e.,  a  system  can  provide  an  interoperation,  but  not  accept  it),  then  directional 
interoperability  measurements  (see  next  definition)  must  be  made. 

DEFINITION  (Directional  Interoperability  Measurement):  If  two  systems 

a'  =  , a"  =  [a are  instantiated  with  directional 

interoperability  characters  X  =  ,  then  m  =  /(cr;,cr;)  is  a 

measure  of  the  directional  interoperability  of  cr'  to  (X . 

3.5.3  Interoperability  Measurement  Modes 
3.5.3. 1  Directional 

All  interoperations  are  either  bi-directional  or  uni-directional.  Bi-directional 
interoperation  implies  =/(cr',cr'')  =  =/(cr'',cr')  whereas  uni-directional 
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interoperation  implies  that  ^  (e.g.,  cr'  is  a  transmit  only  system  and  cr"  is  a 

receive  only  system). 

3.5.3.2  Self 

Self  interoperability  is  defined  as  w  =  /  (cr,  cr)  and  is  usually  assumed  to  be  zero. 

Self  interoperability  implies  an  interoperation  originating  at  the  system,  exiting  the 
system  boundary  and  then  accepted  back  through  the  boundary.  An  example  of  this  is  a 
network  loopback  “ping”  in  which  a  computer  attempts  to  detect  its  own  IP  address  on 
the  local  network. 

3.5.3.3  Pure 

Pure  interoperability  is  a  measure  of  the  interoperability  of  two  systems  whose 
instantiations  are  aligned  only  with  each  other,  hence  their  interoperability  measure  is 
pure,  or  unencumbered  by  other  systems’  interoperability  characters.  In  other  words, 

l-S”!  =  2 .  Pure  interoperability  is  measured  for  performance  or  cost  analysis  reasons, 
among  others.  The  following  example  illustrates  the  concept. 

Given  5'  =  {xj,5'2},  X  =  [x^,X2,x^^  (alibi-directional),  Ce|]Rn[0,9]|,  S 

instantiated  as  E  =  |crj,cr2}  =||l,2,3},|4,5,6}| ,  and  I  =  (with  Minkowski 

parameter  set  to  r  =  2 ),  an  interoperability  matrix  is  obtained  (16)  which  shows  that  the 
interoperability  of  o'^,o'2  is  relatively  low.  This  was  to  be  expected  as  the  values  of  both 

system  instantiation’s  character  states  are  all  well  below  =  9 .  No  self- 
interoperability  was  assumed,  so  the  diagonal  of  M  was  assigned  a  value  of  0. 
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M  = 


0  0.259 


(16) 


0.259  0  J 

If  a  third  system  is  added  to  S ,  bringing  with  it  additional  interoperability 
characters,  then  expectedly,  a  third  instantiation  is  added  to  Z  and  the  size  of  M 

increases,  but  the  value  of  /(crj,cr2)can  change  as  well  because  the  context,  or  basis,  of 

the  interoperability  measurement  has  changed.  This  phenomenon  is  called  contextual 
interoperability  and  is  addressed  next. 

3.5.3.4  Contextual 

Contextual  interoperability  is  a  measure  of  the  interoperability  of  two  systems 
whose  instantiations  have  been  aligned  with  at  least  one  other  system  possessed  of  one  or 
more  different  characters  not  used  to  characterize  the  initial  two  systems.  In  other  words, 
it  is  the  measure  of  the  interoperability  of  two  systems  in  the  context  of  a  larger  set  of 
systems.  By  increasing  the  size  of  S ,  the  number  of  characters  in  X  generally  increases 
(although  not  necessarily  so),  thus  providing  a  basis  for  a  more  precise  interoperability 
measurement.  This  is  analogous  to  making  a  length  measurement  with  a  measuring  tape 
with  only  two  markings,  versus  using  one  with  a  hundred  markings.  More  markings 
yield  a  more  precise  basis  for  the  length  measurement. 

Applying  this  idea  and  taking  the  example  given  in  the  previous  section,  add  to 

S  where  is  instantiated  as  cTj  ={3,7, 8, 9}  and  X  =  \x^,X2,Xj^,x^^ .  Aligning  the  three 
system  instantiations  yields  the  following  complete  instantiation  of  S . 
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'1  2  3  o' 

2=  4  5  6  0  (17) 

3  7  8  9 

Again,  using  /  =  as  the  interoperability  function,  and  assuming  n  =  4, 

r  =  2,  =  9 ,  and  no  self-interoperability,  the  interoperability  matrix  in  (1 8)  is 

obtained. 

0  0.207  0.162' 

M=  0.207  0  0.276  (18) 

0.162  0.276  0 

By  adding  a  new  system  instantiation  to  2  ,  which  increased  the  number  of 
characters  used  to  instantiate  S ,  the  interoperability  measurement  of  and  S2  becomes 
more  precise,  changing  from  /  =  0.259  to  /  =  0.207  .  This  drop  in  the  interoperability 
measurement  was  expected  because  the  interoperability  of  Xj  and  Xj  is  now  being 

measured  in  the  context  of  which  not  only  adds  a  new  interoperability  character  to  the 
model  but  exhibits  much  higher  character  state  values  than  the  other  two  system 
instantiations.  In  the  context  of  the  expanded  model,  5'j  and  ^'2  still  appear  very  similar 

(i.e.,  the  Modified  Minkowski  Similarity  function  shows  their  similarity  is  MMS  =  0.83  ) 
but  their  overall  interoperability  measurement  is  penalized  by  their  low  interoperability 
character  state  values  ( w  =  0.292 ).  In  the  context  of  the  very  capable  ,  the 

interoperability  measurement  of  5'j  and  ^'2  drops.  While  this  result  might  be  considered 
non-intuitive,  it  is  nevertheless  correct.  Indeed,  s^  and  .^2  are  not  less  interoperable 
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because  ^3  is  added  to  S ,  but  their  interoperability  measurement  becomes  more  precise 


because  of  the  infusion  of  the  additional  characters  associated  with  ^3 .  It  can  be 

postulated  that  as  the  number  of  characters  used  to  instantiate  S  approaches  infinity,  then 
the  interoperability  measurements  of  the  systems  in  S  approach  perfect  precision. 

3.5.3.5  Time-variant 

Interoperability  is  generally  time  variant.  For  example,  atmospheric  effects  due  to 
the  changes  from  night  to  day  will  degrade  the  optical  interoperability  of  reconnaissance 
satellites  and  ground  targets.  Similarly,  the  directional  interoperability  of  an  attacker  and 
his  target  may  increase  as  the  attacker  has  ingressed  long  enough  to  come  in  range  of  the 
target.  Finally,  end-to-end  computer  interoperability  may  improve  or  diminish  with 
changes  in  network  congestion  tied  to  worker  shift  changes,  lunchtime  usage,  etc.  There 
are  two  distinct  methods  of  modeling  time-variant  interoperability.  The  first  method 
creates  a  time-continuous  basis  for  the  interoperability  measurement  in  which  systems  are 
instantiated  using  interoperability  characters  which  themselves  are  functions  of  time. 
Hence,  the  resulting  interoperability  measurement  is  also  a  function  of  time.  The  second 
method  is  a  discrete  method  in  which  a  series  of  instantiations  are  created  which 
represent  “snapshots”  in  time.  The  series  of  interoperability  measurements  tied  to  these 
instantiations  represents  a  sampled  time- varying  interoperability  measurement.  Time- 
variant  interoperability  measurements  can  be  directional,  self,  pure,  contextual, 
confrontational,  collaborative,  direct,  or  in-direct  interoperability  measurements.  A  time- 
variant  interoperability  measurement  can  be  equivalent  to  an  activity-phased 
measurement  if  the  activities  occur  in  time  sequential  fashion.  Time  variant 


57 


interoperability  measurements  are  useful  in  determining  if/when  interoperability  lapses 
cause  associated  degradation  in  operational  effectiveness. 

3.5.3.6  Constrained  Upper  Bound  on  Interoperability  Measurement 

It  is  not  possible,  or  even  desirable,  for  all  systems  to  interoperate  with  each  other, 
let  alone  interoperate  perfectly.  Furthermore,  for  those  systems  which  ought  to 
interoperate,  there  are  often  many  limitations,  specific  to  the  operational  process,  which 
prevent  them  from  interoperating  at  their  full  potential.  Some  of  these  limitations  are 
physical  (e.g.,  electromagnetic  interference),  operational  (i.e.,  rules  of  engagement),  or 
reliability  related  (i.e.,  mission  capability),  among  many  others.  Therefore,  in  order  to 
manage  expectations  on  the  final  interoperability  measurement,  it  is  useful  to  define  a 
realistic  upper  bound  on  the  interoperability  measurement,  called  the  constrained  upper 
bound.  This  constrained  upper  bound  on  the  interoperability  measurement  admits  that 
the  best  possible  interoperability  measurement  must  be  less  than  m  =  1  due  to  these 
various  degradations  and  limitations.  The  constrained  upper  bound  on  the 
interoperability  measurement  is  calculated  by  first  determining  the  operational  process- 
specific  interoperability  limiting  factors,  then  by  building  an  interoperability  model 
which  accommodates  those  limitations.  This  model  includes  all  interoperability 
characters  the  set  of  systems  could  conceivably  implement  with  their  character  states  set 
to  their  best  possible  value  in  light  of  the  predetermined  limitations.  The  difference 
between  the  constrained  upper  bound  on  the  interoperability  measurement  and  the  current 
interoperability  measurement  is  called  the  interoperability  gap  and  represents  the  trade 


58 


space,  design  spaee,  or  funding  spaee  in  which  process  or  system  ehanges  can  occur,  to 
the  end  of  improving  operational  effectiveness. 

3.5.3.7  Collaborative 

Collaborative  interoperability  is  the  type  of  interoperability  most  understood  by 
people  today.  For  example,  in  the  Department  of  Defense  the  term  interoperability  is 
usually  associated  with  the  idea  of  service,  joint,  and  allied  systems,  units,  and  forees 
working  together  to  mutual  advantage  as  opposed  to  friendly  (blue)  systems 
interoperating  with  adversary  (red)  systems.  Similarly,  in  the  eivilian  sector,  an  engineer 
might  eonsider  interoperability  to  be  a  property  of  how  well  the  various  systems  he 
operates  or  designs  interface  or  provide  and  accept  services  from  each  other.  Written 
another  way,  eollaborative  interoperability  is  the  interoperability  between  friendly,  or 
blue  systems.  Collaborative  interoperability  also  earries  a  connotation  that  if  one  system 
provides  something,  another  system  gives  something  baek  in  exchange.  In  other  words, 
the  systems  interoperate  in  a  collaborative  fashion  for  mutual  benefit  or  to  achieve  a 
shared  goal.  Applying  traditional  military  terminology,  collaborative  interoperability  can 
also  be  ealled  blue-to-blue  interoperability.  The  interoperability  measurement  method 
presented  in  this  Chapter  provides  a  methodical  means  of  measuring  the  eollaborative 
interoperability  of  all  types  of  systems  interoperating  in  all  kinds  of  ways. 

It  is  important  to  note  however,  that  not  all  interoperations  are  collaborative. 
Indeed,  collaborative  interoperability  often  supports  the  goals  of  the  operational  proeess 
but  does  not  direetly  implement  them.  For  example  eollaborative  forms  of 
interoperability  such  as  linguistic,  technical,  logistics,  and  cultural  interoperability  of 
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coalition  forces  and  systems  enabled  the  ouster  of  Iraq  from  Kuwait  during  operation 
DESERT  STORM,  however  they  did  not  directly  cause  the  ouster — that  was  done  by 
confrontational  interoperability  methods  such  as  attack  operations.  In  general,  if  the 
operational  process  implies  any  form  of  opposition  (i.e.,  an  oppositional  or 
confrontational  operational  process)  between  systems  (e.g.,  negotiation,  attack, 
greediness,  pushing,  removing,  limiting,  and  preventing,  among  others),  then  another 
type  of  interoperability,  called  confrontational  interoperability  can  be  measured.  This 
type  of  interoperability  measurement  is  powerful  because  it  can  be  related  directly  to 
operational  effectiveness  without  discrete  event,  or  other  types  of  simulation. 

3.5.3.8  Confrontational 

A  unique  contribution  of  this  research  is  the  announcement  and  explanation  of 
confrontational  interoperability,  which  is  the  interoperability  of  friendly  (blue)  and 
adversary  (red)  forces.  Examples  of  confrontational  interoperability  include  the  actions 
of  a  friendly  (blue)  jammer  to  degrade  the  communications  of  adversary  (red)  force 
command  and  control  systems,  the  efforts  of  two  negotiation  teams  attempting  to  resolve 
an  issue,  environmental  activity  which  inhibits  technology  (e.g.,  gravity  vs  a  rocket,  wind 
vs  an  airplane,  oxidation  vs  a  metal  bridge,  or  sunspot-generated  electromagnetic 
interference  vs  radios),  and  marketing  aimed  at  attracting  business  for  one  company 
while  preventing  it  for  another,  among  others.  In  a  confrontational  operational  process, 
one  set  of  systems  tries  to  achieve  advantage  or  effectiveness  over  the  remaining  systems. 
Many  military  processes,  such  as  time  critical  targeting,  psychological  operations,  or 
defensive  counter  air,  are  inherently  confrontational.  Indeed,  for  these  processes 
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confrontational  interoperability  should  receive  foeus  as  it  describes  the  ability  of  friendly 
(blue)  systems  to  eontrol  adversary  (red)  systems  and  to  prevent  reciprocation — an 
inherently  military  concept. 

Confrontational  interoperability  implies  and  underpins  effeets-based  operations  in 
which  all  plarming,  preparation,  exeeution,  and  assessment  of  an  operation  eoncentrates 
on  the  desired  effeets  on  the  enemy.  Everything  that  does  not  contribute  toward 
achieving  the  desired  effect  is  considered  irrelevant  and  anything  that  eould  hinder  the 
effeet  is  eliminated.  For  example,  in  a  time  critieal  targeting  operation,  the  goal  is  not  to 
have  a  high  degree  of  eollaborative  interoperability  of  friendly  systems  (this  might  be 
helpful  or  could  be  damaging  to  the  operation),  but  to  have  a  high  degree  of  directional 
confrontational  interoperability  from  friendly  (blue)  to  adversary  (red)  systems.  In  other 
words,  the  goal  is  to  ensure  that  blue  systems  are  able  to  destroy  red  systems.  Indeed,  for 
certain  operational  proeesses,  eollaborative  interoperability  ideally  is  minimized  and 
confrontational  interoperability  is  maximized.  For  example,  during  stealth  operations,  a 
single  stealthy  aireraft  may  conduct  a  bombing  operation  without  support  aircraft,  while 
maintaining  radio  silence,  and  while  flying  without  the  benefit  of  active  radar. 

While  too  mueh  eollaborative  interoperability  in  any  type  of  operational  process 
might  result  have  degrading  effeets,  such  as  information  overload,  in  a  confrontational 
operational  process,  a  high  degree  of  directional  confrontational  interoperability  from 
friendly  to  adversary  systems  is  always  desired  as  it  will  likely  result  in  decisive  suecess. 
Furthermore,  in  the  eontext  of  a  eonfrontational  operational  process,  when  the  statement 
is  made  that  eollaborative  interoperability  must  be  improved,  the  implieation  is  that  it 
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must  be  improved  in  order  to  increase  operational  effectiveness  (i.e.,  improve 
confrontational  interoperability).  Some  have  made  the  dangerous  assumption  that  the 
more  interoperable  friendly  systems  and  forces  are  with  each  other  the  more  operational 
success  they  will  enjoy.  This  is  often  false.  For  example,  fighter  pilots  have  warned 
against  information  overload  in  the  cockpit  which  might  distract  them  from  their  main 
tasks.  Similarly,  the  ability  of  mid-  and  senior-level  managers  to  interoperate  with  each 
other  and  their  subordinates  via  e-mail  is  both  a  benefit  and  a  detractor  if  misused. 
Information  overload,  distraction,  and  inefficiencies,  among  other  detractors,  are  the 
result  of  too  much  collaborative  interoperability.  The  relationship  between 
confrontational  interoperability  and  operational  effectiveness  is  described  in  succeeding 
sections  and  demonstrated  in  Chapter  4. 

3.6  Interoperability  Impact  on  Operational  Effectiveness 

General  Hal  Homburg  quipped  in  2004  that  he  looks  forward  “to  the  day  where 
we  can  convince  a  surface-to-air  missile  that  it’s  a  Maytag  in  a  rinse  cycle,  making  it 
irrelevant  to  combat.”  (Tirpak,  2004:31)  This  statement  implies  a  desire  to  control  the 
enemy.  Stated  another  way,  General  Homburg  desires  friendly  (blue)  force  operational 
advantage  resulting  from  improved  friendly  (blue)-to-adversary  (red)  directional 
interoperability  and  degraded  adversary  (red)-to-ffiendly  (blue)  directional 
interoperability.  Sun  Tzu  encapsulates  this  concept  by  stating  “The  clever  combatant 
imposes  his  will  on  the  enemy,  but  does  not  allow  the  enemy’s  will  to  be  imposed  on 
him.”  (Giles,  1910:VI-2)  The  following  axiom  gives  a  sufficient  condition  which  relates 
directional  confrontational  interoperability  to  operational  advantage. 
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AXIOM  (Operational  Advantage):  hettht  subscripts  B,R  refer  to 


friendly  (blue)  and  adversary  (red)  forces  respectively.  Given  a  set  of 
systems  S  =  {Sg,Sjf]  instantiated  as  X  =  then  a  sufficient 

condition  for  friendly  (blue)  force  operational  advantage  over 
adversary  (red)  force  is  for  all  pairs  (crg,cr^),  /( 

assuming  2  completely  characterizes  S . 

While  the  Operational  Advantage  axiom  states  that  if  the  directional 
interoperability  of  all  friendly  (blue)  toadversary  ( red)  systems  exceeds  that  of  adversary 
(red)  to  friendly  (blue),  then  operational  advantage  is  obtained,  it  is  important  to 
emphasize  that  this  is  a  sufficient,  but  not  a  necessary  condition.  For  example,  if  a  set  of 
friendly  (blue)  systems  includes  a  home  base  and  a  bomber  ,  while  the  set  of 

adversary  (red)  systems  consists  of  an  integrated  air  defense  system  CFj^g  and  a  target 

,  then  probably  the  home  base  does  not  need  to  have  operational  advantage  over  the 

enemy  target  in  order  for  the  operation  to  be  effective,  however  the  bomber  must  have 
operational  advantage  over  both  the  air  defense  system  and  the  target.  Thus,  in  this 
limited  example  it  might  be  postulated  that  a  necessary  condition  for  friendly  (blue)  force 
operational  advantage  operational  advantage  over  adversary  (red)  force  is  that 

■^ ( ^  I I ,(7g^ .  Unfortunately,  these 

necessary  conditions  are  not  always  possible  to  define,  and  generally  discrete  event  or 
some  other  type  of  simulation  is  required  in  order  to  determine  the  minimal  set  of 
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systems  which  must  enjoy  operational  advantage  over  each  other  in  order  to  ensure 
operational  success. 

As  MoOEs  quantify  operational  advantage,  applying  the  Operational  Advantage 
axiom,  the  impact  of  interoperability  on  operational  effectiveness  can  be  described.  This 
important  result  is  given  below  as  the  Interoperability  Impact  on  Operational 
Effectiveness  axiom,  which,  like  the  Operational  Advantage  axiom,  describes  a  sufficient 
condition.  The  Interoperability  Impact  on  Operational  Effectiveness  axiom,  1)  applies  to 
confrontational  interoperability  (noting  that  collaborative  interoperability  between 
friendly  systems  contributes  to  confrontational  interoperability  between  opposing  forces’ 
systems),  2)  requires  that  the  MoOE  be  written  as  a  diametric  pair,  and  3)  demands  that 
the  set  of  systems  be  instantiated  by  a  complete  set  of  interoperability  characters  X 
which  describe  all  interoperations  related  to  the  diametric  pair.  For  example,  if  is  the 

percent  of  red  targets  destroyed  and  is  the  percent  of  red  targets  protected,  then  X 

must  characterize  all  interoperations  between  all  systems  in  S  which  contribute  to  the 
destruction  and  protection  of  red  targets. 

AXIOM  (Interoperability  Impact  on  Operational  Effectiveness) :  Let  the 
subscripts  B,R  refer  to  friendly  (blue)  and  adversary  (red)  forces 

respectively.  Given  a  set  of  systems  S  =  [Sg,Sj^]  characterized  by  X 

and  instantiated  as  S  =  }  and  a  diametric  MoOE  0  =  [0g,0j^] , 

then  if  X  completely  characterizes  all  interoperations  related  to 
0^,0 jf  then  a  sufficient  condition  for  Og  >  Og  is  that  friendly  (blue) 
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systems  have  Operational  Advantage  over  adversary  (red)  systems,  or 
in  other  words  friendly  (blue)-to-adversary  (red)  directional 
interoperability  exceeds  adversary  (red)-to-friendly  (blue)  directional 

interoperability  for  all  pairs  (cr^,cr^)  (i.e., 

^(cr^ , C7^  ) , / (cr^ , )  >  / (o-^ , ). 

3.7  Architecture  and  Interoperability 

Numerous  enterprise,  system,  and  operational  architectures  have  been  created 
over  the  past  decade  using  the  DoD  Architecture  Framework  (DoDAF).  Many,  if  not  all, 
of  the  organizations  which  created  these  architectures  are  also  interested  in 
interoperability.  This  is  apparent  in  the  heavy  focus  on  describing  needlines,  information 
exchanges,  and  system  functions  as  well  as  in  documenting  the  technical  standards 
required  for  communication  and  net-centric  operation.  Unfortunately,  only  some  of  the 
key  elements  required  to  perform  an  interoperability  measurement  (e.g.,  operational 
process  and  set  of  systems)  can  be  extracted  from  a  candidate  DoDAF  architecture 
description.  Additionally,  the  Core  Architecture  Data  Model  (CADM)  used  as  a  template 
for  storing  DoDAF  architecture  descriptions  (DoD,  2007c)  has  only  a  portion  of  the 
structure  necessary  to  store  the  required  elements  for  an  interoperability  measurement- 
focused  architecture.  The  CADM  designers  were  definitely  considering  interoperability 
when  they  designed  the  model.  For  example,  they  included  a  seldom  used  field  called 
interoperabilityLevelCode  intended  to  hold  a  LISI  level.  However,  using  CADM  to 
store  the  key  elements  given  in  Table  8  would  be  an  inefficient  force-fit  at  best.  For 
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example,  CADM  provides  for  storing  systems  in  multiple  places  (i.e.,  what  is  called  a 
system  in  the  context  of  this  dissertation  is  called  a  system,  sub-system,  system  node, 
operational  node,  and  organization  within  the  CADM  model).  Possibly  even  more 
concerning  are  fields  in  CADM  which  simultaneously  store  interoperability  measurement 
elements  and  non-interoperability  measurement  elements  (e.g.,  operational  activities, 
system  functions,  needlines,  and  information  exchanges  stored  in  a  CADM  database 
could  be  interoperability  characters,  but  not  necessarily  so).  Thus  it  is  concluded  that 
DoDAF  and  CADM  in  their  current  forms  (version  1.5)  are  incapable  of  storing  or 
representing  an  architecture  whose  purpose  is  interoperability  assessment  and  analysis. 

Table  8  Interoperability  measurement  key  elements 

_ Key  Element _ 

Purpose 

Operational  Process 
Measure  of  Operational  Effectiveness 
Set  of  Systems 

Set  of  Interoperability  Characters 
States  of  Interoperability  Characters 
System  Instantiation 
Interoperability  Function 
Interoperability  Measurement 


DoDAF  and  CADM  are  currently  unable  to  accommodate  the  elements  of  Table 
8,  but  it  is  possible  to  extend  them  to  do  so.  An  interoperability-focused  redesign  of 
CADM  is  outside  the  scope  of  this  dissertation,  however,  embracing  the  motivations  of 
DoDAF  version  2.0  development  (architecture  with  a  purpose),  a  set  of  views  are 
proposed  specifically  for  the  purpose  of  interoperability  assessment  (Table  9).  The  views 
proposed  in  the  extension  are  numbered  so  as  to  be  roughly  analogous  to  existing  DoDAF 
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views.  In  the  table,  the  name,  contents,  format,  and  current  DoDAF  analogue  view  are 
given  for  each  proposed  view. 

IV- 1  Interoperability  Purpose  and  Assumptions  is  a  text-  document,  augmenting 
the  AV-1,  which  describes  the  purpose  of  the  interoperability  analysis,  the  associated 
MoOE,  and  critical  assumptions  such  as  which  interoperability  function  was  used  to 
make  the  interoperability  measurement.  It  also  includes  the  final  analysis  made  based 
upon  the  IV-4  view. 

IV-2  Interoperability  Graph  visually  depicts  the  systems  and  their  interoperations. 
A  mathematical  definition  follows. 

DEFINITION  (Interoperability  Graph):  Given  a  set  of  systems  S 

instantiated  as  S  =  |  over  a  set  of  interoperability  characters 

X  with  character  states  C ,  let  G  be  a  directed  multigraph  in  which 
V{^G)  =  S  and  E[G^  is  a  set  of  edges  such  that  for  all  characters 

X.  e  X ,  there  exists  a  directed  edge  from  s'  to  s”  labeled  with  the 

name  of  that  character  if  s'  is  able  to  provide  that  interoperation  and 
s"  is  able  to  accept  it. 

IV-3  System  Instantiation  ( E )  is  a  system  to  interoperability  character  matrix 
containing  states  of  the  characters,  IV-4  Interoperability  Measurement  is  the  matrix  of 
pairwise  system  interoperability  measurements  M ,  and  IV-5  Operational  Process  (i.e.,  an 
OV-5)  is  a  SysML  activity  diagram  modeling  the  operational  process.  A  simple 
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interoperability  arehiteeture  is  given  in  Table  10  to  illustrate  the  five  interoperability 
arehitecture  produets. 
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_ Table  9  Interoperability  extension  to  DoDAF _ 

View _ Contents _ Format  DoDAF  Analogue 


.  o  G 

on  X  cj 

^  ^  r  • 

C3  Cj_| 


c  p  o  ^  q-) 

►5  I  <C  Q 

C/^C/^^GOGOOcOC/i  ^ 

I  I  I  I  I  I  I  c 

o^> 

Oc/^c/^Ogogogogoc/^^O 


^  dj 
C  PhP3 


p 

(D 

o 

^— » 

c 

0> 

o 

^— » 
CD 

0> 

g 

1— I 

c/3 

>. 

0> 
^— » 

c/3 

o 

c/3 

c/3 

O 

o 

0> 

o 

o 

ts 

’»-» 

05 

o5 

o3 

o5 

c/5 

c/5 

c/5 

c/5 

c/5 

41^ 

& 

2 

a 

o> 

C/3 

C/3 

O 

*^— » 

c3 

0> 

o 

>. 

*^— » 

§ 

c/3 

o 

;-H 

Plh 

2 

0> 

a 

1— I 

c3 

;-H 

o> 

a 

CIh 

g 

a 

‘'g 

o 

;-H 

o> 

0> 

^— » 

c/3 

o 

0> 

cd 

dj 

^— » 
a 

1— 1 

C/5 

^— » 
a 

l-H 

P- 

o 

(N 

m 

U-) 

> 

> 

> 

> 

1— I 

1— 1 

l-H 

HH 

c^ 


Table  10  Example  interoperability  architecture 


NAME:  Precision  Strike  Interoperability  Measurement 

PURPOSE:  To  predict  operational  success  by  application  of  the  Interoperability 
Impact  on  Operational  Effectiveness  axiom. 

INTEROPERABILITY  FUNCTION:  /  =  Sim, 


Bin 


MoOE:  O  =  {Og,Og\  ,  is  “target  destroyed?”  and  (9^  is  “target  protected.” 
ANALYSIS:  Assuming  the  critical  interoperation  is  PSP-TGT ,  then  because 
I  [a pgp,  a >  I  (^apgp,  a pgp)  then  it  is  assumed  that  Op  >  Op. 
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3.8  Summary 

The  general  method  of  interoperability  measurement  presented  in  this  chapter  is 
foundational  and  weaves  together  the  best  ideas  from  extant  literature  with  key  missing 
elements  described  in  this  research  (e.g.,  system  interoperability  characters,  character 
states,  system  instantiations,  system  similarity,  operational  advantage,  and  others).  The 
result  is  a  flexible  method  suitable  for  measuring  the  collaborative  and  confrontational 
interoperability  of  all  types  of  systems  interoperating  in  all  types  of  ways.  Because  the 
method  is  grounded  in  an  operational  process,  the  interoperability  measurement  is  not 
abstract,  but  mathematically  related  to  the  operational  effectiveness  of  the  confrontational 
operational  process.  The  flexibility  of  the  method  supports  the  instantiation  of  systems  at 
any  level  of  abstraction,  with  resultant  interoperability  measurements  at  any  desired  level 
of  precision.  In  the  method  of  this  chapter,  numerous  weaknesses  of  extant  methods  are 
resolved.  For  example,  no  longer  is  interoperability  measurement  limited  to  specific 
types  of  systems  or  interoperations,  no  longer  is  an  interoperability  measurement  an 
abstract  measure  divorced  from  the  operational  circumstances  in  which  the  interoperation 
occurred,  and  no  longer  are  interoperability  measurements  restricted  to  the  precision  of  a 
limited  scale  or  the  accuracy  of  a  limited/outdated  set  of  attributes.  In  short,  the  method 
of  this  chapter  defines  a  basic  theory  of  interoperability  measurement. 
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4.  Analysis  and  Results 


One  key  way  to  ensure  effectiveness  is  to  ensure  that  our  systems  are  interoperable. 

— General  Lester  Lyles 


4.1  Overview 

The  interoperability  measurement  method  of  Chapter  3  is  foundational  and 
broadly  useful.  By  applying  the  method,  the  interoperability  of  organizations,  coalitions, 
weapon  systems,  technology,  philosophies,  the  environment,  and  uncountable  other 
entities  can  be  measured.  While  it  is  not  possible  to  provide  an  example  for  all  possible 
applications  (Chapter  3,  Table  4)  of  the  interoperability  measurement  method,  several  are 
given  in  this  chapter  to  demonstrate  its  application. 

First,  it  will  be  shown  that  maturity  model  (leveling)  methods  are  a  special  case  of 
the  more  general  method  of  Chapter  3.  Specifically,  the  Organizational  Interoperability 
Model  (OIM)  will  be  modeled  using  the  method  of  Chapter  3  and,  using  the  same 
example  given  in  Clark  &  Moon  (2001),  the  interoperability  of  coalition  forces  will  be 
measured  and  the  results  compared  to  that  of  Clark  &  Moon.  Second,  to  demonstrate  the 
relationship  of  interoperability  with  operational  effectiveness,  the  Interoperability  Impact 
on  Operational  Effectiveness  axiom  will  be  applied  to  a  Suppression  of  Enemy  Air 
Defenses  (SEAD)  problem.  Finally,  the  time  variance  of  interoperability  will  be  explored 
through  a  Precision  Strike  application. 

4.2  Application:  Coalition  Interoperability 

Technological  interoperability  has  been  commonly  discussed  in  other  research, 
often  focusing  on  network  information  technology  standards,  however,  other  types  of 
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interoperability  are  often  more  important.  For  example,  the  US  air  strikes  against  Libya 
in  1986  not  only  highlighted  equipment  interoperability  problems  (i.e.,  the  Navy  lacked 
HAVE  QUICK  radios  which  the  Air  Force  possessed),  but  more  especially  joint 
procedural  interoperability  problems  between  the  Navy  and  Air  Force.  (Clark  &  Moon, 
2001)  Similarly,  NATO  forces  experienced  secure,  tactical  voice  communication 
problems  in  Kosovo,  not  because  of  lack  of  proper  radios,  but  also  because  of  procedural 
interoperability  problems.  (Nutwell  &  Price,  2000).  Finally,  Lieutenant  General  Cevic 
Bir,  the  commander  of  United  Nations  Operations  Somalia  (UNOSOM  II)  in  1993 
remarked  that  interoperability  was  a  major  problem  in  every  phase  of  his  coalition 
operation.  (Bir,  1997)  Joint  and  coalition  interoperability  must  be  addressed,  not  just  at 
the  technical  level,  but  also  at  the  organizational  level.  Coalition  interoperability 
measurements  can  focus  the  commander’s  efforts  on  improving  joint  and  coalition 
warfighting  effectiveness  by  increasing  the  interoperability  of  coalition  forces. 

Clark  &  Jones  recognized  the  importance  of  coalition  interoperability  and 
described  a  maturity  model,  called  the  Organizational  Interoperability  Model  (OIM), 
which  describes  a  framework  of  coalition  interoperability  attributes  and  levels.  (1999) 
Their  model  is  based  on  the  structure  of  the  Department  of  Defense  Levels  of  Information 
Systems  Interoperability  (LISI)  model  and  is  given  in  Table  11. 
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Table  1 1  Organizational  interoperability  maturity  model  (OIM)  (Clark  &  Moon,  2001) 


Preparedness 

Understanding 

Command 

Style 

Ethos 

Level  4 
(Unified) 

Complete 
normal  day-to- 
day  working 

Shared 

Homogeneous 

Uniform 

Detailed 
doctrine  and 
experience  in 
using  it 

One  chain  of 

Shared  ethos 

Level  3 
(Combined) 

Shared 

communications 
and  knowledge 

command  and 
interaction  with 
home 

organization 

but  with 
influence  from 
home 

organization 

Separate 

Shared 

Level  2 

( Collaborative ) 

General 
doctrine  in 
place  and  some 
experience 

Shared  comms 
and  shared 
knowledge  about 
specific  topics 

reporting  lines 
of 

responsibility 
overlaid  with  a 
single 

purpose;  goals, 
value  system 
significantly 
influenced  by 
home 

command  chain 

organization 

Electronic 

Separate 

Level  1 
(Cooperative) 

General 

guidelines 

comms  and 
shared 

reporting  lines 
of 

Shared  purpose 

information 

responsibility 

Level  0 

No 

Voice  comms  via 

No  interaction 

Limited  shared 

(Independent) 

preparedness 

phone,  etc. 

purpose 

The  usefulness  of  the  OIM  model  was  demonstrated  when  Clark  &  Moon  used  it 
to  analyze  the  International  Force  East  Timor  (INTERFET)  coalition  sent  to  enforce 
peace  in  East  Timor  in  1999.  (2001)  The  coalition  consisted  of  forces  from  Australia 
(lead  nation),  the  United  States,  New  Zealand,  Thailand,  Phillipines,  and  Republic  of 
Korea  among  others.  Using  after-action  reports  pertaining  to  the  operation,  Clark  & 
Moon  were  able  to  apply  their  model  to  determine  qualitatively  that  the  highest  levels  of 
interoperability  occurred  between  Australia,  the  United  States,  and  New  Zealand,  that 
Thailand  and  the  Phillipines  enjoyed  a  lower  level  of  interoperability  with  Australia,  and 
that  all  other  interoperations  were  at  the  lowest  level. 
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Clark  &  Moon  assessed  the  interoperability  of  INTEREFET  coalition  forces  with 
respect  to  the  Australian  Deployable  Joint  Force  (ADJF)  standard,  as  implemented 
according  to  the  American,  British,  Canada,  and  Australia  (ABCA)  coalition  operations 
handbook  (COH).  Obviously,  not  all  INTERFET  member  nations  were  capable  of 
adhering,  or  even  desired  to  adhere  completely  to  this  standard,  and  their  interoperability 
levels,  as  assessed  by  Clark  &  Moon,  reflected  this.  For  example,  Clark  &  Moon  stated 
that  with  respect  to  the  Preparedness  attribute,  the  Thai  forces  were  scored  at  level  1  with 
respect  to  the  standard,  while  the  United  States  was  scored  at  level  2.  Similarly,  in  Clark 
&  Moon’s  aggregate  interoperability  measurement,  when  they  state  that  the 
interoperability  of  the  US  and  the  Republic  of  Korea  was  level  1,  it  is  to  be  understood 
that  this  assessment  was  made  1)  in  context  of  the  INTERFET  operation,  and  2)  with 
respect  to  the  ADJF -ABCA  COH  standard.  For  the  OIM  model,  level  0  infers  lack  of 
interoperability  and  level  4  infers  full  compliance  with  the  standard. 

Hypothesis:  Maturity  model  (leveling)  interoperability  assessment  methods,  such 
as  the  OIM  model,  can  be  shown  to  be  a  special  case  of  the  general  interoperability 
method  presented  in  Chapter  3.  To  demonstrate,  the  method  of  Chapter  3  can  be  used  to 
measure  the  interoperability  of  INTERFET  coalition  forces  and  to  arrive  at  the  same 
conclusions  as  those  made  by  Clark  &  Moon  when  they  applied  the  OIM  model  to  the 
INTERFET  operation. 

4.2.1  The  Special  Case  of  the  Maturity  Model  Method 

A  maturity  model  (or  leveling)  interoperability  method  such  as  LISI  or  OIM 
defines  a  set  of  entities,  interoperability  attributes  (usually  limited  to  one  to  four 
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X  which  characterize  the 


entities,  and  the  levels  are  states  of  those  characters  C .  For  each  of  the  entities  modeled, 
the  maturity  model  methods  take  the  lowest  interoperability  level  of  all  the  attributes,  and 
call  it  the  generic  interoperability  G.  of  that  entity.  Then  the  interoperability 

measurement  for  a  pair  of  entities  is  called  the  expected  interoperability,  and  is  defined  as 
the  lesser  of  the  two  entities’  generic  interoperability  measurements.  In  other  words. 

Expected  Interoperability  =Min{G^,G2) .  For  example,  in  Table  12,  the  expected 

interoperability  assessment  of  Entity  #1  and  Entity  #2  is  1  (i.e.,  the  minimum  of  the  two 
generic  interoperability  assessments).  Using  the  terminology  from  Chapter  3,  it  can  be 

said  that  the  maturity  models  use  an  interoperability  function  I  =  Min{G^ , G2 )  to 

calculate  the  interoperability  of  a  pair  of  systems  .  Thus,  the  maturity  model  method  is 
shown  to  be  a  limited  case  of  the  general  method  of  Chapter  3  in  which  entities  equate  to 
systems  S ,  attributes  equate  to  characters  X ,  levels  equate  to  character  states  C ,  and 
minimum  common  generic  interoperability  is  used  as  an  interoperability  function  /  . 


Table  ] 

2  Example  OIM  interoperability  assessments 

Preparedness 

Understanding 

Command  Style 

Ethos 

Generic  Interoperability 

Entity  #1 

3 

2 

2 

1 

1 

Entity  #2 

4 

4 

3 

3 

3 

4.2.2  INTERFET  Coalition  Interoperability  Results  and  Method  Comparisons 

Clark  &  Moon  applied  the  OIM  model  to  assess  the  interoperability  of  coalition 
forces  participating  in  the  1999  INTERFET  operation.  (2001)  Drawing  from  their  paper. 
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systems,  interoperability  eharacters,  and  charaeter  states  ean  be  assigned  as  in  (19),  (20), 
and  (21). 

5  =  { (75,  US,  NZ,  Thai,  Phil,  ROK)  ( 1 9) 

X  =  I  Preparedness,  Understanding,  Command  Style,  Ethos^  (20) 

C  =  {0,1, 2, 3, 4}  (21) 

While  the  OIM  authors  limited  themselves  to  four  attributes  (eharacters)  and  five 
levels  (states),  using  the  method  of  Chapter  3,  their  model  could  have  been  expanded  to  a 
much  larger  number  of  characters  and  states,  which  would  result  in  a  better 
characterization  of  the  coalition  forces  and  a  more  precise  interoperability  measurement. 
Indeed,  after  the  original  publication  of  the  OIM  model  (Clark  &  Jones,  1999),  other 
researchers  debated  the  number  and  descriptions  of  the  attributes  of  the  OIM  model. 
Although  the  final  version  of  the  OIM  model  remained  limited  to  a  4-attribute,  5-level 
model,  at  least  35  sub-attributes  were  further  defined.  (Fewell  &  Clark,  2003)  The 
method  of  Chapter  3  could  easily  accommodate  these  35  sub-attributes  as  additional 
characters.  Although  these  35  additional  characters  help  an  analyst  assign  an 
interoperability  level  to  each  attribute,  by  not  addressing  them  as  individual  attributes, 
fidelity  is  lost  from  the  model,  and  their  contribution  is  effectively  averaged  out. 
Extracting  from  Clark  &  Moon,  the  set  of  interoperability  characters  are  defined  in  Table 
13. 
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Table  13  Explanation  of  coalition  characterization  X 

Preparedness 

How  well  does  the  nation  adhere  to  ADJF  standards  as  implemented  by 
ABCA-COH  doctrine  and  training? 

Understanding 

How  well  does  the  nation  share  information  and  knowledge  according  to 

ADJF  practice  and  ABCA-COH  guidelines? 

Command  Style 

How  well  does  the  nation  delegate  and  share  roles  according  to  ADJF  and 
ABCA-COH  guidelines? 

Ethos 

How  well  does  the  nation  seek  to  assist  the  East  Timorese  and  to  maintain 
their  relationship  with  Indonesia? 

S  is  instantiated  as  in  (22)  according  to  Clark  &  Moon’s  descriptions  of  member 


nations’  participation  in  the  INTERFET  coalition  operation.  Although  their  paper  gives 
enough  information  to  have  used  more  precise  real-valued  character  states,  integer  states 
were  used  to  maintain  consistency  with  their  model.  One  decimal  precision  is  given  in 
the  interoperability  measurement  (23)  to  illustrate  the  improved  measurement  fidelity. 


Selecting  /  =  as  the  interoperability  function,  with  =  4 ,  r  =  2,  and 

n  =  4 ,  the  following  coalition  interoperability  measurement  M  results. 
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If  the  interoperability  measurement  M  is  sealed  from  its  eurrent  [0,l]  scale  to  the 


OIM  [0,4]  scale,  then  the  measurement  (24)  can  be  compared  to  Clark  &  Moon’s 
original  results  (Table  14). 


^scaled 


AUS 

US 

NZ 

Thai 

Phil 

ROK 

AUS 

2.3 

2.3 

2.3 

1.0 

1.0 

0.9 

US 

2.3 

2.3 

2.3 

1.0 

1.0 

0.9 

NZ 

2.3 

2.3 

2.3 

1.0 

1.0 

0.9 

Thai 

1.0 

1.0 

1.0 

1 

0.9 

0.8 

Phil 

1.0 

1.0 

1.0 

0.9 

1 

0.8 

ROK 

0.9 

0.9 

0.9 

0.8 

0.8 

0.8 

(24) 


T able  14  Clark  &  Moon's  original  INTERFET  interoperability  measurements 
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Clark  &  Moon  noted  the  best  coalition  interoperability  among  the  US,  Australia, 
and  New  Zealand  (OIM  2)  and  the  worst  among  Thailand,  the  Phillipines,  and  the 
Republic  of  Korea  (OIM  0).  Similar  measurements  (24)  result  from  the  method  of 
Chapter  3,  but  with  more  accuracy  and  precision.  For  example,  whereas  the  OIM  model 
scored  the  interoperability  of  the  Republic  of  Korea  with  Australia  as  a  zero,  meaning  the 
nations  were  operating  completely  independently  of  each  other,  the  method  of  Chapter  3 
gives  a  more  accurate  result  of  0.9.  An  OIM  score  of  zero  indicates  the  two  nations  I) 
had  no  level  of  preparedness  to  operate  in  a  coalition  together,  2)  had  no  interaction 
amongst  their  commanders  and  forces,  3)  were  limited  to  telephone  communication,  and 
4)  shared  a  common  purpose  only  in  a  limited  fashion.  However,  Clark  &  Moon’s  paper 
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indicates  that  although  the  Koreans  did  have  preparation  issues,  they  1)  did  jointly  attend 
briefings  and  planning  meetings,  2)  understood  at  least  half  of  the  material  presented  at 
those  briefings  and  meetings,  3)  received  taskings  from  HQ  INTERFET,  but  operated  in 
their  own  area  of  responsibility,  4)  had  personal  contact  between  commanders,  and  5) 
were  not  willing  to  participate  in  all  aspects  of  the  INTERFET  operation,  but  strongly 
supported  the  humanitarian  aspect  of  the  operation.  Considering  that  an  OIM  score  of 
one  indicates  1)  preparation  was  made  by  learning  general  guidelines  (not  met  by 
Koreans),  2)  understanding  is  obtained  through  electronic  communication  and  shared 
information  (partially  met  by  Koreans),  3)  command  is  implemented  through  separate 
lines  of  responsibility  (met  by  Koreans),  and  4)  the  ethos  of  the  operation  is  shared 
(partially  met  by  Koreans),  it  seems  reasonable  to  assume  that  the  Australian-Korean 
interoperability  score  should  probably  be  somewhere  between  zero  and  one.  Thus,  the 
Chapter  3 -derived  measurement  of  0.9  is  appropriate  and  more  precisely  and  accurately 
reflects  the  true  interoperability  of  the  Republic  of  Korea  with  Australia  than  the 
assessment  originally  given  by  Clark  &  Moon. 

4.2.3  Analysis  of  INTERFET  Interoperability  Measurements 

INTERFET  coalition  interoperability  M  is  shown  graphically  in  Figure  7.  It  can 

be  seen  that  among  INTERFET  member  nations  were  three  clusters  [^AUS,US,NZ^ , 

\Thai,PhU^ ,  and  [ROK^  .  Expectedly  the  nations  with  Western-type  philosophies,  and 

presumably  more  familiar  with  ADJF  standards  as  implemented  by  the  ABCA-COH, 
enjoyed  a  high  degree  of  interoperability  with  each  other,  but  less  so  with  the  Asian 
nations  and  vice  versa.  Coalition  interoperability  could  improve  in  the  future  among 
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these  Western  and  Eastern  nations  if  common  philosophies  on  doctrine,  training, 
information  sharing,  delegation,  and  cultural  values  and  goals,  acceptable  to  both  East 
and  West,  are  agreed  upon,  practiced,  and  implemented  prior  to  future  operations. 


Figure  7  INTERFET  coalition  interoperability 
The  lack  of  coalition  interoperability  between  the  Western  and  Eastern  nations 
participating  in  INTERFET  manifested  itself  in  the  fact  that  “the  Thais,  South  Koreans, 
and  Filipinos  had  their  own  areas  of  operation. .  .and  conducted  their  own  operations.” 
(Clark  &  Moon,  2001:32)  Similarly,  the  “divergent  nature  of  the  operational 
philosophies  of  the  participating  countries”  was  one  of  the  “most  difficult  aspects  of 
assembling  and  maintaining  the  coalition.”  (Ibid)  Furthermore,  some  of  the  coalition 
officers  “only  understood  half  of  what  was  said  at  briefings  and  conferences  and. .  .the 
Australians  were  unaware  of  this.”  (Ibid:33) 

4.3  Application:  Suppression  of  Enemy  Air  Defenses  (SEAD) 

The  following  application  further  demonstrates  the  interoperability  measurement 
method  of  Chapter  3,  explores  confrontational  interoperability,  and  illustrates  the 


81 


Interoperability  Impact  on  Operational  Effectiveness  axiom  which  states  that  improved 
friendly  (blue)-to-adversary  (red)  directional  interoperability  combined  with  degraded 
adversary  (red)-to- friendly  (blue)  interoperability  results  in  higher  operational 
effectiveness. 

Hypothesis:  Applying  the  Interoperability  Impact  on  Operational  Effectiveness 
axiom,  it  can  be  shown  that  operational  effectiveness  of  the  SEAD  mission  is  improved 
by  1)  the  addition  of  friendly  (blue)-force  precision  strike  and  electronic  attack  capability 
(i.e.,  increased  friendly  (blue)-to-adversary  (red)  interoperability)  and  2)  the  addition  of 
friendly  (blue)-force  stealth  (i.e.,  decreased  adversary  (red)-to-friendly  (blue) 
interoperability). 

SEAD  is  defined  by  JP  1-02  Department  of  Defense  Dictionary  of  Military  and 
Associated  Terms  as  “activity  that  neutralizes,  destroys,  or  temporarily  degrades  surface- 
based  enemy  air  defenses  by  destructive  and/or  disruptive  means.”  (JP  1-02,  2008:523) 

In  this  application,  the  definition  is  further  refined  to  include  only  activity  which  destroys 
enemy  air  defenses  by  destructive  means.  An  operational  process  for  this  application  is 
given  in  Figure  8  and  is  based  upon  the  targeting  process  given  in  JP  3-60  Joint  Targeting 
and  AFDD  2-1.9  Targeting.  (2007;  2006) 
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CJCSM  3500. 04D  UniversalJoint  Task  List  task  OP  3.2.4  “Suppress  Enemy  Air 
Defenses”  measure  Ml.  (2006)  This  MoOE  can  be  written  as  the  diametric  pair  given  in 
(25)  which  obeys  the  relationship  (9g  +0^  =  1 . 


0  =  {0„0,}  = 


[ Percent  of  enemy  air  defenses  destroyed,] 
[Percent  of  enemy  air  defenses  protected  J 


(25) 


Typical  SEAD  systems  are  associated  with  the  activities  and  decisions  of  the 
operational  process  (Figure  8)  and  are  given  in  (26).  The  ISR  system  performs  the  Find, 
Fix,  and  Track  activities,  the  AOC  system  performs  the  Target,  Assess,  and  Reattack? 
activities  and  decision,  and  the  PSP  (precision  strike  package)  system  performs  the 
Engage  activity.  Two  enemy  IADS  systems  are  targets  for  the  mission.  An  operational 
view  of  the  mission  is  given  in  Figure  9. 
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^  =  {5'^,^^}  =  {{HB,ISR,A0C,PSP},{IADS,JADS^}] 


(26) 


In  order  to  apply  the  Interoperability  Impact  on  Operational  Effectiveness  axiom, 
the  characterization  X  of  S'  must  include  all  interoperability  characters  which  describe 
every  interoperation  (collaborative  and  confrontational)  between  systems  in  S  related  to 
O .  In  other  words,  all  interoperability  characters  related  to  the  destruction  and  protection 
of  the  IADS  systems  must  be  included  in  X  .  This  hierarchical  set  of  directional 
interoperability  characters  X  is  given  in  Table  15.  In  order  to  ensure  the  set  of 
interoperability  characters  X  chosen  for  the  SEAD  application  in  this  section  is 
complete  and  authoritative,  they  have  been  methodically  identified  and  extracted  from 
Joint  Publications  and  Air  Force  Doctrine  Documents  related  not  just  to  suppression  of 
enemy  air  defenses,  but  to  Joint  and  Air  Force  operations  in  general.  The  top  level  of  the 
hierarchy  is  the  set  of  joint  operational  functions  given  in  JP  3-0  Joint  Operations  (2006) 
and  the  second  level  is  a  pertinent  subset  of  the  operational  functions  of  air  and  space 
power  given  in  AFDD  1  Air  Force  Basic  Doctrine.  (2003)  Lower  levels  of  the 
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interoperability  character  hierarchy  have  been  extracted  from  JP  3-0  Joint  Operations 
(2006),  AFDD  2  Operations  and  Organization  (2007),  JP  3-01  Countering  Air  and 
Missile  Threats  (2007),  AFDD  2-1  Air  Warfare  (2000),  AFDD  2-1.1  Counterair 
Operations  (2002),  JP  3-13.1  Electronic  Warfare  (2007),  AFDD  2-5.1  Electronic 
Warfare  (2002),  AFDD  2-9  Intelligence,  Surveillance,  and  Reconnaissance  Operations 
(2007),  JP  3-60  Joint  Targeting  (2007),  AFDD  2-1.9  Targeting  (2006),  and  JP  3-13.4 
Military  Deception.  (2006)  In  order  to  maintain  the  ability  to  assess  the  value  of  future 
capabilities  (e.g.,  precision  ground  attack  and  electronic  attack),  those  interoperability 
characters  are  included  in  X  as  well,  but  their  states  are  zeroed  out  in  the  initial  system 
instantiation  2  .  An  explanation  of  the  interoperability  characters  in  X  is  given  in  Table 
16. 


Table  15  SEAD  hierarchical  characterization  X 


1 

C2 

Intel 

] 

"ires 

2 

Comm 

ISR 

Counterair  (CA) 

Info  Ops  (lO) 

3 

Blue 

Red 

Detect 

OCA 

DCA 

EW 

Infl.  Ops 

4 

Target 

Blue 

Red 

Ground 

EA 

MILDEC 

5 

Cluster 

Precision 

Barrage 

Reactive 
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_ Table  16.  Explanation  of  SEAD  characterization  X _ 

Interoperability  Character  Source  Explanation 

C2  JP  3-0  command  &  control/be  commanded  &  controlled 

C2.  Comm  Ibid  communicate(T)/communicate(R) 


a  ^ 

(D 

"3  a 

T  ^ 


QJ  ^  QJ 

o  g  o 

‘b  §  *s 

B  ^  B 
u  ^ 


(i>  (i>  (i>  y 

^  ^  ^  c3 


(N 

o  o  Q 
rn  rn  Q 

Ph  Ph  ^ 


^  m  m 


^  O 

B 

a  ^ 

^  a 

o  ^ 

o  cd 

^'g 

o  o 

o  o 

cd  a 

c3  d 

H-»  H-» 

ts  ts 

d  d 

(N 

(N 

Q 

o  Q 

Q 

cA  Q 

Ph 

r.  IH 

< 

Qq  Q> 


q  q 

P^’ 

K  ^  ^ 


3  3 

o  o 

3  5 


^  ^  ^  ^ 


q  q 


precision  guided  munitions 

Fires.  CA.DCA  JP  3-01  attack  threats  to  airborne  systems  after  weapon  employment/be  attacked  as  a 

_ AFDD  2-1.1  threat  to  airborne  systems  after  weapon  employment _ 


Interoperability  Character  Source  Explanation 

Fires. lO  JP  3-0  attack  via  info  operations/be  attacked  via  information  operations 

AFDD  1 

Fires.IO.EW  JP  3-0  attack  via  info  operations  using  electronic  warfare/be  attacked  via  info 


00 


Assign  the  set  of  states  of  X  as  absence/presence  states  C  =  {0,1}  then  the 

instantiation  of  S  is  given  as  Z  in  Table  17.  Although  the  joint  operational  functions 
Movement  &.  Maneuver  ,  Protection ,  and  Sustainment  are  included  in  the  instantiation 
for  completeness,  they  have  been  assigned  zero  states  as  their  functionality  is  not  critical 
to  the  following  analysis. 


Table  17.  SEAD  instantiation  Z 
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If  the  interoperability  function  /  =  Simg^^  is  chosen,  then  assuming  no  self- 
interoperability,  the  resulting  directional  interoperability  measurements  are  given  in  (27). 


For  this  application,  the  appropriate  analysis  of  M  is  the  comparison  of  friendly 
(blue)-to-adversary  (red)  and  adversary  (red)-to-friendly  (blue)  interoperability  (i.e., 
confrontational  interoperability)  to  the  end  of  applying  the  Interoperability  Impact  on 
Operational  Effectiveness  axiom  to  determine  if  the  friendly  (blue)  systems  will  enjoy 
operational  effectiveness  over  the  adversary  (red)  systems.  Four  friendly  (blue)- 
adversary  (red)  system  pairs  are  possible  and  must  be  considered,  HB  <->  IADS , 

ISR  <->  IADS ,  AOC  IADS ,  and  PSP  <->  IADS .  The  Interoperability  Impact  on 
Operational  Effectiveness  axiom  (see  Table  18  summary)  shows  only  one  friendly  (blue) 
system  (ISR )  is  operationally  effective  over  the  adversary  (red)  IADS  systems.  Two 
others  (AOC, PSP)  do  not  possess  operational  advantage  over  the  IADS  and  a  third 
(HB )  is  at  a  standoff  (i.e.,  equivalent  directional  interoperability  measurements). 
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Table  18.  SEAD  interoperability  analysis 


Blue-Red  System  Pair 

Analysis 

B>R 

HB  ^  IADS 

/  {HB,  IADS)  =  Kv  =  I  (IADS,  HB) 

No 

ISR^IADS 

I(ISR,IADS)  =  %  >  I  (IADS,  ISR)  =  ^  0^  >  0^ 

Yes 

AOC^IADS 

l(AOC,IADS)  =  Ky  <  I  (IADS,  AOC)  =  Kt  ^ 

No 

PSP  IADS 

I  (PSP,  IADS)  =  Kv  <  I  (IADS,  PSP)  =  %  ^  Og  <  0^ 

No 

The  directional  interoperability  measurements  in  Table  18  indicate  that  adversary 
(red)  targets  are  able  to  be  detected,  but  not  effectively  destroyed  by  friendly  (blue) 
systems.  Additionally,  the  measurement  of  the  directional  confrontational 
interoperability  from  IADS  to  PSP  indicates  that  the  PSP  is  vulnerable  to  destruction 
by  the  IADS  systems.  According  to  the  Interoperability  Impact  on  Operational 
Effectiveness  axiom,  in  order  to  give  friendly  (blue)  systems  operational  effectiveness 
over  adversary  (red)  systems,  friendly  (blue)-to-adversary  (red)  directional 
interoperability  must  exceed  adversary  (red)-to-friendly  (blue)  interoperability.  To  this 
end,  according  to  the  hypothesis  in  the  introduction  to  this  section,  friendly  (blue)-to- 
adversary  (red)  interoperability  will  be  increased  by  adding  precision  strike  and 
electronic  attack  capability  to  the  PSP  system.  Additionally,  adversary  (red)-to-friendly 
(blue)  interoperability  will  be  decreased  by  adding  stealth  capability  to  the  PSP  system 
(manifested  in  the  model  as  an  inability  of  the  IADS  to  detect  the  PSP).  Assuming  that 
adversary  (red)  systems  are  also  capable  of  being  upgraded,  the  ability  to  resist  all  but 
reactive  jamming  will  be  given  to  the  IADS  systems.  The  upgraded  instantiation  is 
given  in  Table  19.  Changes  from  the  original  instantiation  are  highlighted. 
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Table  19.  Upgraded  SEAD  instantiation 


Transmit 

Receive 

lADSl 

IADS2 

lADSl 

IADS2 

CO 

3: 

QC 

5^ 

AOC 

PSP 

CXi 

3: 

QC 

io 

AOC 

PSP 

C2 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

C2.  Comm 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

C2. Comm. Blue 

1 

1 

1 

1 

0 

0 

1 

1 

1 

1 

0 

0 

C2.  Comm.  Blue.  Target 

0 

1 

1 

0 

0 

0 

0 

0 

1 

1 

0 

0 

C2. Comm. Red 

0 

0 

0 

0 

1 

1 

0 

1 

0 

0 

1 

1 

Intel 

0 

1 

0 

0 

1 

1 

0 

0 

0 

1 

1 

1 

Intel.  ISR 

0 

1 

0 

0 

1 

1 

0 

0 

0 

1 

1 

1 

Intel. ISR. Detect 

0 

1 

0 

0 

1 

1 

0 

0 

0 

1 

1 

1 

Intel.  ISR.  Detect.  Blue 

0 

1 

0 

0 

1 

1 

0 

0 

1 

1  ° 

0 

0 

In  tel.  ISR.  Detect.  Red 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

Fires 

0 

0 

0 

1 

1 

1 

0 

0 

0 

1 

1 

1 

Fires. CA 

0 

0 

0 

1 

1 

1 

0 

0 

0 

1 

1 

1 

Fires. CA.OCA 

0 

0 

0 

1 

0 

0 

0 

0 

0 

1 

1 

1 

Fires.  CA.  OCA.Ground 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

1 

1 

Fires.  CA.  OCA.Ground.  Cluster 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

1 

1 

Fires.  CA.  OCA.  Ground.  Precision 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

1 

1 

FIres.CA.DCA 

0 

0 

0 

0 

1 

1 

0 

0 

0 

0 

1 

1 

FI  res.  10 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

1 

1 

FI  res.  10.  EW 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

1 

1 

FIres.lO.EW.EA 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

1 

1 

Fires.  10.  EW.EA.  Barrage 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

Fires.  10.  EW.EA.  Reactive 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

1 

1 

FIres.lO.InflOps 

0 

0 

0 

0 

1 

1 

0 

1 

1 

1 

0 

0 

Fires.  10.  InflOps.  MILDEC 

0 

0 

0 

0 

1 

1 

0 

1 

1 

1 

0 

0 

Movement&Maneuver 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Protection 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Sustainment 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Again  using  /  =  as  the  interoperability  function,  a  set  of  interoperability 

measurements  for  the  upgraded  systems  is  obtained  and  given  in  (28).  As  above, 
changes  from  the  original  interoperability  matrix  are  highlighted. 
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HB 

ISR 

AOC  PSP 

HB 

0 

% 

% 

X 

ISR 

X 

0 

Yii  \ 

Y21 

AOC 

X 

X 

0 

Yii 

PSP 

X 

X 

X 

0 

IADS, 

Vn 

Vn 

Y21 

X 

IADS2 

Vn 

Yii 

Y21 

K 

IADS,  lADS^ 


0  X 


X  0 


After  upgrading  the  PSP  system  with  precision  strike,  electronic  attack,  and 
stealth  and  countering  with  an  adversary  (red)  force  upgrade  of  the  IADS  systems  with 
resistance  to  all  but  reactive  jamming,  then  l[PSP,IADS)  =  %  >  I  (^IADS,PSP)  =  X 


>  Og  .  Hence  the  friendly  (blue)  force  now  has  a  slight  edge  over  the  adversary 

(red)  force,  implying  that  the  percentage  of  adversary  (red)  targets  destroyed  will  be 
greater  than  the  percentage  of  adversary  (red)  targets  protected.  Thus,  the  original 
hypothesis  of  this  application  is  confirmed.  Finally,  it  is  interesting  to  note  that  one 
element  of  friendly  (blue)-to-friendly  (blue)  interoperability  (i.e.,  collaborative 

interoperability)  decreased  as  a  result  of  the  system  upgrades.  Specifically,  I  {^ISR,PSP) 
decreased  from  ^7  to  ^7 .  The  interpretation  of  this  is  that  due  to  the  addition  of  stealth 
capability  the  PSP  system  is  also  less  detectable  by  the  friendly  (blue)  force  ISR 
system. 


4.4  Application:  Precision  Strike 

Time  variance  of  the  interoperability  of  a  set  of  systems  is  caused  by  progression 
through  the  activities  and  decisions  of  the  operational  process,  by  time  variant  characters. 
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or  by  random  effects.  Interoperability  decreases  due  to  time  variance  are  not  always  bad. 
For  example,  a  process  may  call  for  a  certain  system  to  be  periodically  turned  off,  which 
causes  its  interoperability  with  other  systems  to  drop  to  zero.  On  the  other  hand,  some 
interoperability  decreases  are  undesirable,  such  as  those  due  to  time  varying 
electromagnetic  interference.  It  is  useful  to  analyze  interoperability  with  respect  to  time 
in  order  to  highlight  process  bottlenecks,  discover  previously  unknown  environmental 
impacts,  or  to  determine  minimum  required  interoperability  to  meet  operational  goals  to 
the  end  of  optimizing  monetary  investment. 

The  following  application  illustrates  the  time  variance  of  interoperability 
measurements  by  repeated  application  of  the  method  of  Chapter  3  at  various  stages  of  the 
Precision  Strike  mission  and  demonstrates  that  perfect  interoperability  of  all  systems  at 
all  stages  of  a  mission  is  not  desired  or  necessary,  but  that  appropriate  levels  of 
interoperability  should  be  achieved  at  the  appropriate  times. 

Hypothesis:  The  interoperability  of  Precision  Strike  systems  varies  during 
different  mission  time  periods.  Furthermore,  if  the  constrained  upper  bound  on  the 
interoperability  of  each  system  pair  is  achieved  in  each  time  period,  then  the  sufficient 
condition  for  operational  effectiveness  given  by  the  Interoperability  Impact  on 
Operational  Effectiveness  axiom  can  be  relaxed,  yet  still  be  appropriately  applied  to 
predict  operational  effectiveness.  In  other  words,  during  a  specific  time  period  some 
friendly  (blue)-to-adversary  (red)  and  adversary  (red)-to-fi:iendly  (blue)  operational 
advantage  can  be  ignored  if  it  is  not  pertinent  to  that  time  period. 
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In  this  application  (Figure  10),  a  penetrating  strike  package  (PSP)  attacks  its 
target  kinetically  after  being  escorted  part  way  to  the  target  by  a  modified  escort  jammer 
(MEJ).  A  static  baseline  interoperability  measurement  is  first  made,  then  interoperability 
is  measured  at  four  different  time  periods,  (to)  prior  to  the  PSP  and  MEJ  crossing  the 
forward  edge  of  the  battle  area  (FEBA),  (ti)  when  the  PSP  and  MEJ  are  between  the 
FEBA  and  the  missile  engagement  zone  (MEZ),  (t2)  while  the  PSP  is  over  the  target 
(within  the  MEZ),  and  (ts)  after  the  PSP  attacks  the  target  and  is  egressing  the  MEZ. 


The  precision  strike  operational  process  (Figure  1 1)  is  derived  from  the  following 
use  case.  “A  PSP  launches  from  home  base  and  proceeds  towards  its  target  accompanied 
from  base,  across  the  FEBA  and  up  to  the  MEZ  by  a  MEJ,  which  jams  enemy  radar  and 
communications  signals  detected  by  a  stand-off  intelligence,  surveillance,  and 
reconnaissance  (ISR)  platform  orbiting  on  the  friendly  side  of  the  FEBA.  The  PSP 
crosses  into  the  MEZ,  leaving  the  MEJ  outside  the  MEZ  to  orbit  and  jam,  proceeds  to  the 
target,  destroys  it  kinetically,  and  quickly  egresses,  recovering  on  a  safe  route.” 
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Figure  1 1  Precision  strike  operational  process 
Finally,  define  the  diametrie  MoOE  for  the  precision  strike  operational  process  is 

0  =  [0g,0j^}  where  Og  is  “target  destroyed”  and  is  “target  protected.” 

4.4.1  Static  Unconstrained  Interoperability  Model 

For  comparison  purposes,  a  static  interoperability  model  is  provided  first  which  is 
later  perturbed  to  demonstrate  time-variant  interoperability.  Given  a  set  of  systems 
S  =  {PSP,MEJ,IADS,ISR,HB,TGT]  let  each  system  Sj  be  characterized  by  a  set  of 

interoperability  characters  X  =  (29)  where  Xj,and  are  directional 

(transmit/receive)  interoperability  characters.  Let  the  set  of  interoperability  character 
states  be  given  as  absence/presence  states  C  =  {0,l} . 


X  =  < 


{Comm.EM  (T),  Detect. EM  .Radar(T),  Attack.EM  Jam(T),] 
{^Attack. KM  .Ground  (T),  Attack. KM  .Air  (T)  j 

\Comm.EM  {R), Detect. EM  .Radar{R),  Attack.EM  .Jam{R), 
[Attack. KM  .Ground  (R),  Attack. KM  .Air  {R) 


(29) 


Assuming  no  time,  space,  or  other  constraints  (e.g.,  IADS  radar  has  unlimited 
reach,  MEJ  jams  continuously  regardless  of  position,  etc.)  then  S  is  instantiated  as  S 
(30). 
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110  10  110  0  1 

1  1  1  0  0  1  1  0  0  0 

0  10  0  10  110  0 

J  0  0  0  0  1  1  0  0  0 

1000010000 
0  000000010 

Using  /  =  Sinig.^  as  the  interoperability  function,  and  assuming  no  self¬ 
interoperation,  the  interoperability  measurement  M  is  given  by  (31). 

PSP  MEJ  IADS  ISR  HB  TGT~ 

0  I  i  I  i  i 

MEJ  0  f  f  i  0 

M=  IADS  j  \  0  \  0  0  (31) 

ISR  ^  ^  0  0  10 

//fill  0  10  0 

TGT  0  0  0  0  0  0 

The  interoperability  measurement  M  shows  the  direct,  contextual  interoperability 
(both  collaborative  and  confrontational)  of  all  system  pairs  in  S .  The  average 
interoperability  of  5*  is  M  =  0.1 3  .  Applying  the  Interoperability  Impact  on  Operational 
Effectiveness  axiom,  the  results  in  Table  20  show  that  while  some  operational  advantage 
is  enjoyed  (e.g.,  PSP  should  be  effective  in  attacking  the  TGT  and  the  MEJ  should  be 
effective  in  jamming  the  IADS ),  the  sufficient  condition  for  operational  effectiveness  is 
not  met.  However,  the  model  raises  some  questions.  For  example,  does  the  PSP  always 
need  to  have  operational  advantage  over  the  IADS  ?  Likewise,  does  the  MEJ  need  to 
have  operational  advantage  over  the  TGT  ?  Similar  questions  can  be  asked  for  the 
(^ISR,IADS)  and  {^ISR,TGT)  pairs.  And  most  likely  HB  requires  no  direct  operational 
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advantage  over  any  enemy  system.  Hence  a  time-variant  interoperability  measurement  is 
indicated.  For  each  time  period,  systems  are  instantiated  according  to  operational  and 
other  constraints  associated  with  that  time  period.  Additionally,  the  interoperability 
measurement  taken  in  each  time  period  is  the  constrained  upper  bound  for  that  period 
because  each  system  is  instantiated  by  character  states  which  represent  the  best  the 
system  can  and  is  expected  to  do  in  that  time  period. 


Table  20.  Precision  strike  static  model  interoperability  analysis 


Blue-Red 
System  Pair 

Analysis 

PSP  ^  IADS 

I(PSP,IADS)  =  y,  <I(IADS,PSP)  =  %^0^<0j, 

No 

PSP^TGT 

I(PSP,TGT)  =  y  >I{TGT,PSP)  =  Q^0^>0^ 

Yes 

MEJ^LADS 

I(MEJ,IADS)  =  %>  i(iads,mej)  =  y^o^>Oj, 

Yes 

MEJ^TGT 

I (MEJ, TGT)  =  0  =  I(TGT, MEJ)  ^0g<0^ 

No 

ISR^IADS 

I(ISR,IADS)  =  0  <  I(IADS,ISR)  =  y^Og<Oj, 

No 

ISR^TGT 

I  (ISR,  TGT)  =  0  =  1  (TGT,  ISR) 

No 

HB  IADS 

I(HB,IADS)  =  0  =  I(IADS,HB) 

No 

HB^TGT 

I  (HB,  TGT)  =  0  =  1  (TGT,  HB)^0g<0^ 

No 

4.4.2  Time  Variant  Interoperability  Model 

Let  S,X,C  remain  unchanged  from  the  static  model  and  modify  the  system 
instantiation  E  for  the  four  time  periods  described  in  Figure  10.  For  each  time  period, 
the  modifications  to  E  are  changes  to  the  states  of  the  interoperability  characters  which 
describe  exactly  what  the  systems  are  expected  to  do  in  that  time  period. 
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4.4.2.1  Prior  to  Crossing  the  FEBA  (tg ) 

At  time  ,  all  friendly  (blue)  systems  are  safe  on  their  own  side  of  the  FEBA. 

Let  be  given  (32)  with  the  assumptions  of  Table  21.  In  this  time  period  it  is 

appropriate  to  assume  that  only  the  ISR  system  should  have  operational  advantage  over 
the  adversary  (red)  systems. 

10''  0  O'’  0  1  O''  0  0  O'' 

1  0"  0^  0  0  1  0®  0  0  0" 

0  1  0  0  0'  0  O'  0  0  0 

1  0  0  0  0  1  1  0  0  0 

1000010000 
00  0  00000  o'  0 

Table  21.  Precision  strike  assumptions 

a)  PSP  not  detectable  by  radar  until  at  the  soonest 

b)  PSP  can’t  kinematically  attack  until  tj_ 

c)  see  a) 

d)  PSP  not  susceptible  to  missile  attack  until 

e)  MEJ  not  detectable  by  radar  until  at  the  soonest 

f)  MEJ  doesn’t  jam  until 

g)  see  e) 

h)  MEJ  not  susceptible  to  missile  attack  until  ^2 

i)  IADS  has  nothing  to  attack  until 

j)  IADS  can’t  detect  anything  until  at  the  soonest 

k)  TGT  can’t  be  kinematically  attacked  until 

Using  /  =  Sinig.^  as  the  interoperability  function,  an  interoperability  measurement 
at  is  obtained  (33).  Average  interoperability  of  S  at  to  is  =  0.072  and  an 
interoperability  analysis  follows  in  Table  22.  The  analysis  shows  that  no  friendly  (blue) 
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systems  enjoy  operational  advantage  over  adversary  (red)  systems  in  the  first  phase  of  the 
mission.  This  is  only  worrisome  in  one  point,  the  ISR  system  is  less  interoperable  with 
the  IADS  than  viee  versa.  In  other  words,  the  ISR  system  may  not  be  able  to  aecurately 
deteet  the  threat  for  the  PSP .  All  other  system  pairs  are  achieving  desired  levels  of 
interoperability  (i.e.,  their  interoperations  are  not  required  until  later  time  periods). 

PSP  MEJ  IADS  ISR  HB  TGT~ 

PSP  0  \  Olio 

MEJ  i  0  0  110 

=  IADS  0  0  0  1  0  0  (33) 

ISR  \  \  0  0  10 

HB  ^  i  0  10  0 

TGT  0  0  0  0  0  0 


Table  22.  Precision  strike  interoperability  analysis  (t^) 


HB  ^  TGT  I  (HB,  TGT)  =  0  =  I  (TGT,  HB)  -^0^<0^  No 

4.4.2.2  Prior  to  Entering  the  MEZ  ( ) 

At  time  ,  the  PSP  and  MEJ  have  crossed  the  FEBA  and  have  entered  enemy 
territory.  Let  be  given  (34)  constrained  by  the  assumptions  of  Table  23.  In  this  time 
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period,  it  is  appropriate  to  assume  that  the  ISR  and  MEJ  systems  should  have 
operational  advantage  over  the  adversary  (red)  systems.  Any  adversary  (red)  system 
operational  advantage  over  the  PSP  system  can  be  ignored  as  the  PSP  system  is  neither 
in  range  to  attack  the  TGT ,  nor  within  the  range  of  IADS  defensive  counter-air  attack. 

1  r  0  0*  0  1  r  0  0 

1  r  1^  0  0  1  F  0  0  O'* 

0  1  0  0  0  0  r  F  0  0 

1  0  0  0  0  1  1  0  0  0 

100001000  0 
000  00000  0'  0 


Table  23.  Precision  strike  assumptions 

a)  PSP  has  small,  but  finite  chance  of  reflecting  IADS  radar  signal 

b)  PSP  can’t  kinematically  attack  until 

c)  PSP  has  small,  but  finite  chance  of  being  hit  by  IADS  radar  signal 

d)  PSP  still  outside  MEZ 

e)  MEJ  has  small,  but  finite  chance  of  reflecting  IADS  radar  signal 

f)  MEJ  turns  on  jamming  in 

g)  MEJ  has  small,  but  finite  chance  of  being  hit  by  IADS  radar  signal 

h)  MEJ  not  susceptible  to  missile  attack  until  ^2 

i)  IADS  has  small,  but  finite  chance  of  detecting  a  target 

j)  IADS  can  be  jammed 

k)  TGT  can’t  be  kinematically  attacked  until 

Using  I  =  SitHg.^  as  the  interoperability  function,  an  interoperability  measurement 
at  is  obtained  (35).  Average  interoperability  of  S  increased  during  to  =0.12 

largely  due  to  more  system  interoperations  occurring  during  this  time  period  than  in  the 
previous  period.  An  interoperability  analysis  follows  in  Table  24.  The  ISR  system’s 
lack  of  operational  advantage  over  the  IADS  system  remains  unchanged,  however,  in  , 
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the  MEJ  is  beginning  to  disrupt  the  IADS  system’s  ability  to  counter  the  inbound  PSP 
(highlighted  in  the  table). 


M.  = 


PSP  MEJ  IADS  ISR  HB  TGT 


PSP 

MEJ 

IADS 

ISR 

HB 

TGT 


0 

2 

5 

i 

5 

i 

5 

i 

5 


1 
5 

2 
5 

0 

0 

0 

0 


i 

5 

0 

0 


0 

0 

0 

0 

0 

0 


Table  24.  Precision  strike  interoperability  analysis  (tj) 


(35) 


Blue-Red 
System  Pair 


Analysis 


0b>0^ 


PSP  ^  IADS 
PSP  TGT 
MEJ  ^  IADS 
MEJ  ^  TGT 
ISR  ^  IADS 
ISR  TGT 
HB  <A-  IADS 
HB^TGT 


I  {PSP,  IADS)  =  y,=I  (IADS,  PSP)  No 

I{PSP,TGT)  =  0  =  I{TGT,PSP)  ^ No 
/  (MEJ,  IADS)  =  %>  I  (IADS,  MEj)  =  y^O^>0^  Yes 
I  (MEJ,  TGT)  =  0  =  I  (TGT,  MEJ)  -^0^<0^  No 

I(ISR,IADS)  =  0  <I(IADS,ISR)  =  y^0^<0^  No 

I(ISR,TGT)  =  0  =  I  (TGT, ISR)  -^0^<0^  No 

I(HB,  IADS)  =  0  =  I  (IADS,  HB)  No 

/  (HB,  TGT)  =  0  =  7  (TGT,  HB)^0^<0^  No 


4.4.2.3  Within  the  MEZ  ( ) 

At  time  ,  the  PSP  is  within  the  MEZ.  Let  be  given  (36)  using  the 

assumptions  in  Table  25.  During  this  key  time  period,  it  is  highly  desired  that  all  friendly 
(blue)  systems  (except  HB )  have  operational  advantage  over  the  adversary  (red)  systems 
as  the  PSP  is  within  the  MEZ  (i.e.,  vulnerable  to  defensive  counter-air  attack). 
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Additionally,  success  of  the  mission  is  contingent  upon  the  friendly  (blue)  foree 
maintaining  operational  advantage  over  the  adversary  (red)  foree  in  this  time  period. 

1  1  0  r  0  1  1  0  0  1* 

1  1  1  0  0  1  1  0  0  0 

0  1  0  0  r  0  1  1  0  0 

1  0  0  0  0  1  1  0  0  0 

100001000  0 
0000  0000  I''  0 

Table  25.  Preeision  strike  assumptions 

a)  PSP  can  attack  TGT 

b)  PSP  vulnerable  to  kinematie  airborne  attack 

c)  IADS  can  attack  airborne  targets 

d)  TGT  vulnerable  to  kinematie  ground  attaek 

Using  /  =  Sinig.^  as  the  interoperability  funetion,  the  interoperability 

measurement  at  ^2  (37)  shows  that  the  average  interoperability  of  S  inereased  slightly  to 

=0.13  during  this  eritical  phase  of  the  mission  indicating  an  increase  in  the  number 

of  interoperations  (i.e.,  increased  operational  intensity).  However,  the  eontinued  laek  of 
operational  advantage  of  the  PSP  over  the  IADS  in  this  time  period  is  espeeially 
eonceming  in  as  the  PSP  is  within  the  MEZ  and  subjeet  to  attaek.  Henee,  the  ISR 
system’s  continued  inability  to  detect  threats  endangers  the  PSP  (i.e.,  a  pop-up  surfaee- 
to-air  missile  system  may  emerge,  yet  remain  undeteeted  by  the  ISR  system).  Assuming 
the  PSP  survives  the  IADS ,  however,  the  PSP  will  likely  be  sueeessful  in  destroying 
the  TGT  as  it  possesses  operational  advantage  over  the  TGT  in  this  time  period. 
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PSP  MEJ  IADS  ISR  HB  TGT 


M. 


PSP  0 
MEJ  f 
IADS  f 
ISR  \ 
HB  i 
TGT  0 


1  1  1  i  1 

5  5  5  5  5 

0  I  I  i  0 

i  0  1  0  0 

i  0  0^0 

i  0  1  0  0 

0  0  0  0  0 


(37) 


Table  26.  Precision  strike  interoperability  analysis  {t^) 


Blue-Red 
System  Pair 

Analysis 

Ob>0„ 

PSP  ^  IADS 

I(PSP,  IADS)  =  %<  I(IADS,  PSP)  =  %^0,<0^ 

No 

PSP  TGT 

I(PSP,TGT)  =  X  >I{TGT,PSP)  =  0 ^ 0^  > 0^ 

Yes 

MEJ  ^  IADS 

I  (MEJ,  IADS)  =  %  >I  (IADS, MEJ)  =  y,^0^>0^ 

Yes 

MEJ  ^  TGT 

I (MEJ, TGT)  =  0  =  I (TGT, MEJ)  -^0^<0^ 

No 

ISR  ^  IADS 

I(ISR,IADS)  =  0  <I(IADS,ISR)  =  y^O^<0^ 

No 

ISR  TGT 

I(ISR,TGT)  =  0  =  I(TGT,ISR)  ^  0^  <  0^ 

No 

HB  IADS 

I(HB, IADS)  =  0  =  I(IADS, HB)  ^  0^  <  (9^ 

No 

HB^TGT 

/  (HB,  TGT)  =  0  =  7  (TGT,  HB)^0^<0^ 

No 

4.4.2.4  Returning  to  Base  (^j) 


During  the  final  time  period,  ,  the  target  has  been  attacked  and  the  PSP  and 

MEJ  are  returning  to  base  and  are  out-of-range  of  the  IADS  system.  The  ISR  system  is 
still  on-orbit  ready  for  the  next  mission  and  the  IADS  system  was  not  attacked,  so  it  is 
still  functioning.  Let  2,  be  given  by  (38)  assuming  the  MEJ  stops  jamming  in  (a).  In 

this  final  time  period,  it  is  appropriate  to  assume  that  only  the  ISR  system  should 
maintain  operational  advantage  over  the  adversary  (red)  systems  (the  IADS  system  in 
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particular)  as  it  is  the  only  aircraft  remaining  on-station  (onstensibly  to  support  the  next 
mission). 

1  1  0  0  0  1  1  0  0  0 

1  1  0““  0  0  1  1  0  0  0 

0  1  0  0  0  0  1  1  0  0 

1  0  0  0  0  1  1  0  0  0 

10  0  0010000 

00  0  0000000 

Using  /  =  Sinig.^  as  the  interoperability  function,  an  interoperability  measurement 

at  is  obtained  (39)  with  an  average  interoperability  of  S  of  =  0.1 16 .  By  design, 

during  this  final  phase  of  the  mission,  no  friendly  (blue)  system  has  operational 
advantage  over  the  adversary  (red)  systems.  Table  27  shows,  however,  that  the  PSP  and 

IADS  systems  appear  to  be  at  a  standoff,  i.e.  l[PSP,IADS)  =  =  I  [IADS,  PSP) .  A 

review  of  which  interoperability  character  state  caused  the  measure  to  be  non-zero 

reveals  that  the  PSP  system  can  still  be  detected  by  the  IADS  system.  However,  the 

PSP  is  outside  the  MEZ,  so,  while  it  can  still  be  detected,  it  is  safe  from  an  IADS  attack. 

In  other  words,  the  PSP  makes  no  effort  to  hide  itself  from  detection  since  it  can’t  be 

attacked  anyway.  Similar  logic  applies  to  the  MEJ . 

PSP  MEJ  IADS  ISR  HB  TGT~ 

PSP  Of  i  I  i  0 
MEJ  f  0  i  I  i  0 

=  IADS  i  I  0  I  0  0  (39) 

ISR  ^  ^  0  0  10 

7/5  11  0  10  0 

TGT  0  0  0  0  0  0 
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Table  27.  Precision  strike  interoperability  analysis  (t^) 


4.4.2.5  Precision  Strike  Conclusions 

An  important  conclusion  can  be  made  as  a  result  of  the  analysis  in  previous 
sections — although  the  sufficiency  condition  given  by  the  Interoperability  Impact  on 
Operational  Effectiveness  axiom  was  relaxed  in  each  mission  time  period  by  neglecting 
some  interoperations  and  demanding  others  critical  to  that  time  period,  the  operational 
effectiveness  concerns  listed  in  Table  28  were  apparent  in  each  mission  time  period. 
Specifically,  the  lack  of  ISR  operational  advantage  over  the  IADS  throughout  all  time 
periods  combined  with  the  IADS  operational  advantage  over  the  PSP  during  ^2  when 

the  TGT  is  to  be  attacked  indicates  that  the  mission  could  likely  not  be  successful. 
Finally,  a  minor  conclusion  that  can  be  drawn  that  because  the  HB  system’s  only  role  is 
to  launch  the  PSP  and  MEJ  aircraft,  once  it  successfully  does  so,  it  can  be  neglected 
throughout  the  remainder  of  the  analysis. 
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Table  28  Precision  strike  operational  effectiveness  concerns 

Time 

Period 

Operational  Effectiveness  Concern 

^0 

ISR  can’t  clearly  detect  IADS 

ISR  can’t  clearly  detect  IADS 

IADS  can  detect  the  inbound  PSP 

^2 

IADS  has  operational  effectiveness  over  the  PSP  which  is  in-range  of  the  IADS 

ISR  can’t  clearly  detect  IADS 

h 

None 

The  operational  effectiveness  concerns  listed  in  Table  28  can  possibly  be 
alleviated  by  improving  the  ISR  detection  capability  (i.e.,  increasing  its  confrontational 
interoperability)  or  by  decreasing  the  PSP ’s  detectability  by  either  adding  stealth  or 
improving  the  MEJ ’s  jamming  ability.  Figure  12  shows  the  change  in  the  directional 
interoperability  of  each  friendly  (blue)  system  with  each  adversary  (red)  system  over  time 
and  Figure  13  shows  the  change  in  average  confrontational  interoperability  (i.e.,  friendly 
(blue)-to-adversary  (red)  vs.  adversary  (red)-to-friendly  (blue)  interoperability)  with  time. 
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Figure  12  Precision  strike  blue  systems  directional  interoperability  with  red  systems 


Figure  13  Precision  strike  average  interoperability  versus  time 
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4.5  Summary 

In  this  chapter,  three  applieations  were  given — Coalition  Interoperability, 
Suppression  of  Enemy  Air  Defenses  (SEAD),  and  Precision  Strike.  In  the  Coalition 
Interoperability  application,  it  was  hypothesized  and  shown  that  maturity  model 
(leveling)  interoperability  measurement  methods  are  a  speeial  case  of  the  more  general 
method  given  in  Chapter  3.  Speeifieally,  it  was  shown  that  attributes  of  models  such  as 
EISI  and  OIM  can  be  used  as  interoperability  charaeters  and  LISI  and  OIM  levels  ean  be 
equated  to  charaeter  states.  The  SEAD  applieation  further  demonstrated  the 
interoperability  measurement  method  of  Chapter  3  and  illustrated  that  the  Interoperability 
Impact  on  Operational  Effectiveness  axiom  ean  be  used  to  relate  interoperability  to  the 
operational  effectiveness  of  a  eonfrontational  operational  proeess.  The  resultant 
interoperability  measurement  ean  be  used  to  identify  areas  for  system  upgrade  or  for 
doctrine,  taeties,  teehniques,  or  proeedures  change.  Finally,  the  Precision  Strike 
application  further  exemplified  the  suffieient  eonditions  given  by  the  Interoperability 
Impact  on  Operational  Effectiveness  and  Operational  Advantage  axioms  and  showed  how 
they  ean  be  relaxed  within  the  bounds  of  operational  and  time  constraints. 
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5.  Conclusion 


Everything  we  do... we  do  with  an  eye  toward  jointness  and  interoperability. 

— K.  Krieg,  recent  USD(AT&L) 

5.1  Conclusions  of  Research 

Measuring  interoperability  has  long  been  considered  unquantifiable  because  of  its 
complex  nature.  (Kasunic  &  Anderson,  2004)  While  interoperability  has  been  defined 
and  described,  it  is  multifaceted  and  permeates  many  disciplines  in  many  ways.  In  fact,  it 
is  reasonable  to  assume  that  interoperations  occur  in  all  human  endeavors.  Previous 
approaches  to  measuring  or  describing  interoperability  relied  upon  problem 
decomposition  and  several  researchers  designed  limited  methods  (Chapter  2)  of 
measuring  specific  types  of  interoperations  of  certain  types  of  entities.  The  result  was  an 
eclectic  set  of  somewhat  related  models  useful  within  limited  spheres.  While  the  problem 
decomposition  method  was  helpful  in  the  short  term  for  certain  applications,  it  prevented 
the  creation  of  a  general  interoperability  measurement  method  because  it  fractured 
“interoperability  thinking”  into  compartments.  The  answer  to  the  problem  could  not  be 
found  by  creating  “a  set  of  compatible  models  that  collectively  address  all  the  dimensions 
of  interoperability”  (Morris,  et.  al,  2004:12)  because  the  set  would  never  be  complete,  but 
by  looking  at  interoperability  holistically,  generally,  and  fundamentally  and  then 
describing  a  flexible  method.  Such  a  method  was  proposed  in  Chapter  3 — a  method 
which  accommodates  all  types  of  systems  and  interoperations,  produces  a  quantitative 
interoperability  measurement  as  realistic  and  precise  as  desired,  and  is  limited  only  by  the 
desires  and  attentiveness  of  those  who  will  use  it. 
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5.2  Recommendations  for  Application 

Chapter  1,  Table  4  lists  approximately  fifty  applieations  of  the  interoperability 
measurement  method  presented  in  Chapter  3.  The  list  is  small  subset  of  a  complete  list, 
containing  military  applications  extracted  from  only  six  DoD  publications.  While  some 
of  the  applications  in  Table  4  pertain  to  technical  system  interoperability,  many  concern 
non-technical  interoperability  or  cross-domain  (i.e.,  mixed  system)  interoperability. 

These  three  areas  are  ripe  for  application  of  the  method  given  in  Chapter  3.  Additionally, 
problems  which  have  not  traditionally  been  viewed  as  interoperability  related  can  also  be 
analyzed  in  a  new  way  with  the  method  of  this  dissertation.  Discussion  follows. 

5.2.1  Technical  Interoperability 

With  some  exceptions,  historically  interoperability  has  largely  been  associated 
with  technical  systems.  It  is  expected  that  this  focus  will  remain  into  the  foreseeable 
future  because  of  the  importance  of  technical  interoperability  (both  collaborative  and 
confrontational)  to  military  concepts  such  as  Network  Centric  Warfare,  Enterprise 
Integration,  and  Warfighting  Transformation.  The  need  to  analyze  technical 
interoperability  will  not  diminish  although  there  will  be  an  increased  emphasis  on 
viewing  technical  interoperability  within  the  context  of  operational  art.  (Alberts,  et.  al., 
2000)  This  interest  in  interoperability  constrained  by  operations  endears  itself  to  the 
method  of  Chapter  3.  Previous  interoperability  assessments,  such  as  the  LISI  profiles 
formerly  required  by  CJCSI  62 12.0 1C  (2003)  can  now  be  repeated  in  the  context  of  the 
operational  process  in  which  the  systems  were  designed  to  function  and  the  measurement 
can  be  made  with  more  fidelity.  Additionally,  technical  interoperability  measurements 
for  new  systems  can  be  made  which  don’t  rely  solely  upon  technical  standards,  but  upon 
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interoperability  characters  associated  with  the  operational  processes  of  newer  concepts 
such  as  Systems-of-Systems  Integration. 

5.2.2  Non-technical  Interoperability 

One  of  the  most  important  types  of  non-technical  interoperability  is  coalition 
interoperability.  The  United  States  often  fights  as  a  coalition  and  because  not  all  of  our 
allies  possess  the  same  advanced  equipment  that  the  United  States  military  does, 
interoperability  of  doctrine,  procedures,  and  culture  rise  in  importance  relative  to 
technical  interoperability.  These  non-technical  types  of  interoperability  facilitate 
integration  and  synchronization  of  coalition  forces.  JP  3-16,  Multinational  Operations 
states  that  “coalition  partners  using  very  different  national  doctrines  will  obviously  have 
problems  harmonizing  their  efforts,  even  if  they  enjoy  a  high  degree  of  technical 
interoperability.”  (2007:111-8)  The  method  in  Chapter  3  can  be  used  to  predict  the  impact 
of  language  and  cultural  differences  among  coalition  forces,  to  innovate  compatible 
tactics,  techniques,  and  procedures,  to  measure  the  usefulness  of  liaison  officers,  or  to 
troubleshoot  coalition  operations  problems.  A  sample  of  this  type  of  analysis  was  given 
in  Section  4.2. 

5.2.3  Cross-domain  Interoperability 

Power  lies  in  the  ability  of  the  interoperability  method  in  Chapter  3  to  measure 
cross  domain  interoperability,  such  as  investigating  how  well  non-technical  systems 
interoperate  with  technical  ones.  For  example,  when  engineers  design  a  human-computer 
interface  (HCI),  they  are  trying  to  optimize  the  interoperability  of  the  human  and  the 
machine.  Asking  the  question,  “how  efficiently  are  the  flight  controls  laid  out?”  is 
equivalent  to  saying  “how  interoperable  is  the  pilot  with  the  airplane?”  The  popular 
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human  factors  engineering  terms,  “designing  for  human  use”  and  “optimizing  working 
conditions”  (Sanders  &  McCormick,  1987:5)  are  statements  of  a  desire  for  non-technical- 
to-technical  interoperability.  Because  the  method  of  Chapter  3  admits  a  set  of  systems  of 
any  type,  this  type  of  mixed-system  interoperability  measurement  is  now  possible. 

5.2.4  Non-traditional  Interoperability 

Applying  the  method  of  Chapter  3,  a  host  of  problems  not  previously  viewed  as 
interoperability  related  can  now  be  looked  at  as  such.  This  means  that  many  old 
problems  can  be  solved  in  a  new  way,  possibly  lending  insight  or  providing  a  means  of 
reporting  progress  not  previously  available.  For  example,  when  studying  the  relations 
between  two  countries,  often  the  disciplines  of  history,  political  science,  diplomacy, 
business,  and  economics  provide  the  tools  for  the  analysis.  If  interoperability  is  a 
relationship  between  two  systems,  then  two  countries  can  be  modeled  as  systems,  and 
instantiated  with  characters  representing  all  the  ways  in  which  the  two  countries  interact. 
The  degree  of  these  interactions  can  also  be  captured.  Thus,  a  measure  of  the 
interoperability  of  the  two  countries  is  a  measure  of  the  quality  of  their  relationship.  If 
separate  interoperability  measurements  are  taken  with  respect  to  cultural  interoperations, 
business  interoperations,  technical  interoperations,  and  others,  then  a  portfolio  is  created 
which  describes  the  full  spectrum  of  relations  between  the  two  countries.  The  resulting 
set  of  measurements  can  be  used  to  drive  policy,  motivate  trade,  encourage  cultural 
interchange,  or  encourage  cooperation  in  technical  development.  The  method  of  Chapter 
3  can  be  used  to  perform  “what  if’  analyses  to  predict  the  efficiency,  cost,  or  other 
ramifications  of  diplomatic,  economic,  or  military  policy  changes. 


112 


5.3  Recommendation  for  Future  Research 


As  a  general  method  for  interoperability  measurement,  this  researeh  might  be  the 
impetus  for  other  researeh  projeets.  Two  topies  immediately  present  themselves.  First, 
while  this  researeh  foeused  on  direet  interoperability  measurement,  many  important 
interoperations  oceur  through  a  translator  system  (i.e.,  indirect  interoperability). 
Although  a  preliminary  method  of  measuring  the  indirect  interoperability  of  systems  was 
given  by  Ford,  et.  al.,  (2007a,  2008b)  this  method  is  not  complete  and  can  be  further 
extended.  Second,  while  this  research  related  confrontational  interoperability  (i.e., 
friendly  (blue)-to-adversary  (red)  system  interoperability)  with  operational  effectiveness, 
no  analogue  was  given  which  associates  the  change  in  friendly  (blue)-to-friendly  (blue) 
system  interoperability  with  changes  in  operational  effectiveness.  Each  of  these  future 
research  areas  is  discussed  in  detail  in  subsequent  sections. 

5.3.1  Indirect  Interoperability  Measurement 

The  method  in  Chapter  3  measures  the  direct  interoperation  of  systems,  however 
an  enhancement  to  the  method  could  possibly  be  made  which  supports  measuring  the 
indirect  interoperability  of  systems.  Indirect  interoperation  infers  that  a  pair  of  systems 
cannot  interoperate  without  the  assistance  of  another  system.  For  example,  m^  2  is  the 

measure  of  the  direct  interoperability  of  Xj  and  ^'2 .  Indirect  interoperation,  on  the  other 

hand,  implies  an  intermediary  system.  In  other  words,  given  5”  =  {xj,5'2,X3} ,  if  Wj  3  =  0 

but  Wj  2  >  0  and  >  0  then  it  may  be  possible  for  5'j  and  ^'3  to  interoperate  indirectly 

(i.e.,  influence  each  other)  via  ^'2  which  is  called  a  translator  system. 
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Alberts  &  Hayes  stated  this  concept  in  a  different  fashion,  noting  three  types  of 
interoperation — common  language,  direct  translation,  and  translation  via  reference 
language.  (2003)  The  first  type,  common  language  interoperation,  equates  to  the  term 
direct  interoperation  in  this  research.  Unfortunately,  Alberts  &  Hayes  used  the 
oxymoronic  term,  direct  translation,  to  refer  to  an  interoperation  that  is  not  direct,  but  in¬ 
direct  with  an  intermediary  providing  translation.  Finally,  their  third  type,  translation  via 
reference  language,  is  just  a  special  case  of  the  first.  Alberts  &  Hayes  used  a  language 
example  in  their  text  to  illustrate  the  three  types  noting  that  common  language  means  that 
two  people  can  converse  in  the  same  language.  Direct  translation  was  exemplified  as  a 
group  of  English  speakers  communicating  with  a  group  of  French  speakers  via  a 
translator  versed  in  both  languages.  Finally,  translation  via  reference  language  was 
illustrated  as  two  groups  speaking  different  languages  who  communicate  with  each  other 
by  speaking  a  third  (reference)  language. 

Being  able  to  measure  the  indirect  interoperation  of  systems  is  important  because 
something  done  by  one  system  may  or  may  not  impact  its  adjacent  neighbors,  but  may 
have  a  drastic  effect  on  a  “distant”  system.  For  example,  an  e-mail  from  one  person  to 
another  might  result  in  no  action,  but  the  same  e-mail  forwarded  from  the  second  person 
to  a  third  could  cause  an  uproar.  In  other  words,  the  first  person  in-directly  interoperated 
with  the  third,  causing  an  effect.  Similarly,  a  ground  system  interoperating  with  the 
Global  Information  Grid  (GIG)  has  the  potential  to  indirectly  interoperate  with  the 
myriad  of  space-  and  airborne  systems  also  networked  to  the  GIG.  This  is  especially  true 
if  the  ground-based  system  is  a  service  provider  (i.e.,  a  weather,  geospatial,  or 
intelligence  providing  system).  Thus,  indirect  interoperability  is  measured  to  analyze 
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distant  effects  as  opposed  to  adjacent  effects.  For  example,  indirect  interoperability 
measurements  might  help  determine  the  impact  of  US  domestic  policy  on  foreign 
countries  or  it  could  be  used  to  measure  the  impact  that  a  mix  of  legacy  systems  and 
technologically  advanced  systems  have  on  each  other  as  documents  are  constantly 
translated  between  them.  The  ability  to  measure  indirect  interoperability  facilitates 
effects-based  action  where  one  system  impacts  distant  systems. 

Ford,  et  al.,  addressed  indirect  interoperability  in  their  i-Score  papers.  (2007a; 
2008b)  They  described  a  single  measure  of  the  interoperability  of  a  set  of  systems 
implementing  one  sequential  pass  through  an  operational  process.  Each  system  was 
viewed  as  a  translator  and  was  assigned  an  interoperability  “spin”  which  indicated  how  it 
interacted  with  the  succeeding  system  in  the  process  (i.e.,  no  translation  needed,  human 
translation  needed,  or  machine  translation  needed).  The  overall  interoperability  score 
was  impacted  each  time  a  translation  (interoperation)  took  place.  It  was  noted  that  each 
time  an  interoperation  occurs,  there  is  a  potential  loss  (change)  in  physical,  syntactic,  or 
semantic  structure  of  the  original  input  to  the  process.  This  loss  only  gets  magnified  each 
time  the  original  input  is  translated.  Hence,  translations  occurring  early  in  the  process 
have  more  potential  possibility  for  change  than  those  occurring  late  in  the  process.  To 
account  for  this,  each  translation  was  given  a  case-specific  penalty  usually  resulting  from 
a  performance  overlay.  In  essence,  the  i-Score  measurement  was  a  measure  of  the 
interoperability  of  the  first  system  in  the  sequence  with  the  last.  More  work  remains  to 
be  done  in  order  to  define  a  general  method  of  measuring  indirect  interoperability. 
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5.3.2  Collaborative  Interoperability  Impact  on  Operational  Effectiveness 

Section  3.6  provided  an  axiom  which  relates  the  impact  of  interoperability  on 
operational  effectiveness.  However,  the  axiom  only  applies  to  confrontational  (friendly 
(blue)-to-adversary  (red)  system)  interoperability.  In  other  words,  it  relates  an 

improvement  in  friendly  (blue)-to-adversary  (red)  directional  interoperability  /(crg,cr^) 

to  an  improvement  in  friendly  (blue)  operational  effectiveness  .  But  the  question 

remains  as  to  the  impact  of  collaborative  (i.e.,  friendly  (blue)-to-friendly  (blue)) 
interoperability  improvements  on  operational  effectiveness.  To  date,  there  is  no  analogue 
to  the  Interoperability  Impact  on  Operational  Effectiveness  axiom  which  relates  a  change 
in  collaborative  interoperability  to  a  change  in  friendly  (blue)  operational  effectiveness. 

It  is  reasonable  to  assume  that  if  collaborative  system  interoperability  /(crg,cr^) 

improves,  that  Og  will  usually  also  increase.  A  recognized  researcher  in  the  area  of 

network  centric  warfare,  John  Garstka,  speaking  specifically  on  interoperations  in  which 
information  is  passed,  wrote  that  there  is  “a  strong  correlation  between  information 
sharing. .  .and  significantly  increased  combat  power.”  (2000: 1)  He  further  states  that 
while  this  is  intuitive  to  the  warfighter,  quantifying  the  relationship  is  “an  analytical 
challenge  of  the  first  order.”  (Ibid:3)  Hence,  he  resorts  to  providing  empirically  observed 
evidence  of  the  relationship  between  a  particular  network  centric  improvement  and 
increased  combat  power  as  gleaned  from  experiments,  exercises,  demonstrations,  and 
real-life  conflicts  to  justify  his  assertion.  (Ibid)  Interestingly,  Keenan  wrote  a  decade 
earlier  that  the  contribution  of  interoperability  initiatives  on  battlefield  effectiveness  can 
be  assessed  quantitatively.  (1988)  He  gave  six  measures  of  effectiveness  (functional  area 
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performance,  personnel  requirements,  systems  cost,  supporting-to-supported  ratio, 
reconstitution  capability,  and  satisfaction  of  CINC’s  priorities),  which,  when  properly 
measured  and  weighted,  can  be  used  to  generate  an  assessment  of  improvement  in 
battlefield  effectiveness.  (Ibid)  Unfortunately,  Keenan’s  six  measures  cannot  be  shown 
to  be  a  complete,  nor  a  correct  list,  making  his  resulting  interoperability  impact 
assessment  a  subjective  measure. 

While  no  simple  and  complete  relationship  between  collaborative  interoperability 
and  operational  effectiveness  has  yet  been  published,  such  an  axiom  could  be  on  the 
horizon.  The  key  to  discovering  it  likely  lies  in  the  area  of  indirect  interoperability 
measurement  discussed  in  the  previous  section.  While  a  change  in  the  interoperation  of 
friendly  (blue)  systems  has  no  direct  impact  on  the  directional  confrontational 
interoperability  of  friendly  (blue)-to-adversary  (red)  systems,  it  most  definitely  has  an 
indirect  impact.  For  example,  Garstka  noted  that  F-15Cs  equipped  with  the  Joint  Tactical 
Information  Distribution  System  (JTIDS)  experienced  a  kill  ratio  2.5  times  higher  than 
that  of  non- JTIDS  equipped  F-I5Cs.  (2000)  In  other  words,  an  improvement  in 
collaborative  (F-I5C-to-E-3  and  F-I5C-to-F-I5C)  interoperability  indirectly  increased 
confrontational  (F-15C-to-Target)  interoperability,  resulting  in  an  improvement  in 
operational  effectiveness.  A  rigorous  method  of  measuring  indirect  interoperability  of 
systems  might  result  in  an  axiom  describing  the  impact  of  collaborative  interoperability 
on  operational  effectiveness. 

While  early  proponents  of  network  centric  warfare  and  other  interoperability 
improvement  initiatives  noted  empirical  evidence  of  improved  operational  effectiveness 
resulting  from  interoperability  improvements,  other  researchers  have  identified  the 
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problem  of  information  overload  (Toffler,  1970)  and  its  effeets.  (Keller  &  Staelin,  1987) 
Noting  that  information-based  interoperations  are  only  a  subset  of  the  overall  set  of 
interoperations,  the  problem  is  extrapolated  from  the  more  specific  term  of  information 
overload  to  the  general  term  of  interoperability  overload.  It  is  postulated  that  a  future 
axiom  relating  collaborative  interoperability  to  operational  effectiveness  will  describe  a 
relation  similar  to  that  graphed  in  Figure  14  which  shows  an  initial  operational  benefit  of 
increased  collaborative  interoperability  followed  by  a  decrease  as  interoperability 
overload  sets  in  with  an  optimum  in  between. 


Operational 

Effectiveness 


Interoperability 


Figure  14.  Hypothetical  relationship  of  collaborative  interoperability  with  operational 

effectiveness 


5.4  Final  Thoughts 

The  Department  of  Defense  has  been  pursuing  interoperability  for  decades. 
While  some  might  argue  that  the  pursuit  has  not  been  aggressive  enough  (GAO,  1987), 
interoperability  of  defense  systems  has  most  definitely  improved.  Whereas  early  goals 
reflected  concerns  about  equipment  commonality  (i.e.,  NATO  standard  equipment),  the 
focus  eventually  moved  at  the  turn  of  the  millennium  to  assessment  of  system-to-system 


118 


interoperability  via  verification  testing.  (Hutchens,  2007)  Architecures  documented 
information  exchange  requirements  which  defined  how  systems  interoperated  with  each 
other  and  the  interoperations  were  described  in  the  Interoperability  Key  Performance 
Parameter  (I-KPP).  (CJCSI  6212. OIB,  2000)  While  the  Department  of  Defense  still 
relies  upon  certification  testing,  the  newer  Net-Ready  KPP  (NR-KPP)  concept  (CJCSI 
62 12.0 1C,  2003)  emphasizes  compliance  with  the  Net-Centric  Operations  and  Warfare 
Reference  Model  (NCOW-RM),  the  use  of  highly  integrated  architectures,  the  definition 
of  interface  profiles,  and  adherence  to  information  assurance  precepts  to  ensure  systems 
are  interoperable  and  bom  joint  (i.e.,  network  centric).  (Hutchens,  2007) 

In  spite  of  all  this  progress,  these  efforts  have  been  focused  on  qualitatively 
describing  technical  interoperability.  The  general  method  of  this  dissertation  provides  a 
means  to  finally  quantitatively  measure  the  interoperability  of  not  only  technical  systems, 
but  non-technical  systems  or  mixed  sets  of  systems.  Because  the  method  draws  upon 
existing  data  already  mandated  by  the  Joint  Capabilities  Integration  and  Development 
System  (JCIDS)  (e.g.,  integrated  architectures,  interface  profiles,  operational  processes, 
measures  of  effectiveness)  it  becomes  an  efficient  extension  to  the  current  state-of-the- 
practice  in  interoperability  assessment.  The  ability  to  put  the  interoperability 
measurement  in  the  context  of  operations  and  determine  the  impact  of  system 
interoperability  on  those  operations  is  an  added  bonus  and  it  is  hoped  that  the  general 
method  of  measuring  system  interoperability  presented  in  this  research  will  greatly 
improve  defense  systems  and  military  operations  for  years  to  come. 
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Al.  A  Selection  of  Interoperability  Definitions  (adapted  from  Ford,  et.  al.,  2007b) 
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A2.  A  Selection  of  Interoperability  Types 


Interoperability  Type 

Source 

Applieation 

(Kasunic  &  Anderson,  2004:34), 
(Kosanke,  2005:4) 

Arehiteeture 

(Curts,  1999:10) 

C4I 

(Kasunic  &  Anderson,  2004:9) 

Cities 

(Kinder,  2002:18) 

Coalition 

(USJFCOM,  2001:48), 

(Fewell  &  Clark,  2003,  p.  1) 

Communieations 

(LaVean,  1980:1448), 

(Kasunic  &  Anderson,  2004:34) 

Conceptual 

(Carney  &  Obemdorf,  2004:18) 

Connected 

(DoD,  1998) 

Constructive 

(Levine,  et.  al.,  2003:5), 

(Carney  &  Obemdorf,  2004:19), 
(Morris,  et  al.,  2004:1 1) 

Constructive 

(Morris,  et  al.,  2004:35) 

Cultural 

(Clark  &  Moon,  2001:2) 

Data 

(Curts,  1999:4), 

(USJFCOM,  2001:30), 

(Kasunic  &  Anderson,  2004:4,  7,  34) 

Domain 

(DoD,  1998) 

Electronic 

(DoDD  2010.6,  1980:Encl.  2,  p.  2) 

Enterprise 

(DoD,  1998), 

(Kosanke,  2005:  8) 

Flexible 

(Clark  &  Moon,  2001:2) 

Force 

(Clark  &  Moon,  2001:1) 

Functional 

(DoD,  1998), 

(USJFCOM,  2001:22), 

(Clark,  2001:2) 

Higher-layer 

(Kasunic  &  Anderson,  2004:34) 

Horizontal 

(Kinder,  2002:27) 

Information 

(Curts,  1999:4) 

Information  Systems 

(DoD,  1998) 

Intra-organisational 

(Kinder,  2002:23) 

Isolated 

(DoD,  1998) 

Joint 

(Leite,  1998:1), 

(USJFCOM,  2001:  49), 

(Kasunic  &  Anderson,  2004:13-14) 

Joint  Information 

(Nutwell,  2000) 

Logistics 

(DoDD  2010.6,  1980:Encl.  2,  p.  3) 

Lower-layer 

(Kasunic  &  Anderson,  2004:34) 

Model 

(Clark  &  Moon,  2001:1) 

Multidatabase 

(Litwin  &  Abdellatif,  1986:1) 

Non-GIG 

(USJFCOM,  2001:29) 

Non-technological 

(Clark  &  Moon,  2001:1) 

Object  Oriented 

(Konstantas,  1993:i) 

Operational 

(Levine,  et.  al.,  2003:6), 

(Carney  &  Obemdorf,  2004:19), 
(Kasunic  &  Anderson,  2004:2), 
(Morris,  et  al.,  2004: 1 1) 

Organizational 

(Clark  &  Jones,  1999:1), 

(Clark  &  Moon,  2001:1) 

Peacetime 

(LaVean,  1980:1450) 

Planned 

(Clark  &  Moon,  2001:3) 

Plug-and-Play 

(USJFCOM,  2001:47) 

Procedure  Oriented 

(Konstantas,  1993:4) 

Process 

(Clark  &  Moon,  2001:2) 

Product-to-Product 

(Kasunic  &  Anderson,  2004:37) 

Programmatic 

(Levine,  et.  al.,  2003:4), 

(Carney  &  Obemdorf,  2004:19), 
(Morris,  et  al.,  2004:1 1) 

Programmatic 

(Morris,  et  al.,  2004:33) 

Public  Administration 

(Kinder,  2002:6) 

Public  Service 

(Kinder,  2002:7) 

Responsive 

(Clark  &  Moon,  2001:2) 

Secure- Voice 

(USJFCOM,  2001:  33) 

Semantic 

(Heiler,  1995:1) 

Specification  Level 

(Wileden,  et.  al.,  1989:74) 

System-of-Systems 

(Morris  et.  al.,  2004: Cover) 

Systems 

(LaVean,  1980:1449), 

(Leite,  1998:  1), 

(Curts,  1999:  3), 

(USJFCOM,  2001:32), 

(Clark  &  Moon,  2001:2), 

(Kasunic  &  Anderson,  2004:  1) 

Sy  stem-to-  System 

(Amanowicz  &  Gajewski,  1996:280), 
(Kasunic  &  Anderson,  2004,  p.  17) 

Technical 

(Clark  &  Jones,  19994), 

(USJFCOM,  2001:22), 

(Clark  &  Moon,  2001:1), 

(Kinder,  2002:25), 

(Carney  &  Obemdorf,  2004:16), 
(Kasunic  &  Anderson,  2004:2) 

T  elecommunications 

(LaVean,  1980:1449) 

Total 

(Curts,  1999:1) 

Transitive 

(Morris,  et.  al.,  2004:28) 

Vertical 

(Kinder,  2002:27) 
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A3. Summary  of  Interoperability  Measurement  Models 

Sixteen  interoperability  measurement  models  (Table  29)  are  surveyed  in 
suceeeding  paragraphs  with  the  main  eontributions  of  eaeh  model  highlighted.  Appendix 
A4  summarizes  the  models’  measurement  formats.  The  sueceeding  review  and  analysis 
is  modified  from  Ford,  et.  al.  (2007b).  Some  model  nicknames  (e.g.,  SoIM,  QoIM, 
MCISI,  and  lAM)  were  not  used  by  the  model  authors,  but  have  been  assigned  by  the 
author  of  this  research  for  ease  of  reference. 

Table  29  Interoperability  measurement  model  publishers  (adapted  from  Ford,  et.  al.. 


2007b) 


Publishing  Organization 

Model 

Defense  Information  Systems  Agency  (DISA) 

SoIM  (’80) 

MITRE  Corporation 

QoIM  (’89) 
LISI  (’98) 

Military  University  of  Technology,  Warsaw,  Poland 

MCISI  (’96) 

Joint  Theater  Air  &  Missile  Def  Org.  (JTAMDO)  Contractor  SIM 

lAM  (’98) 

Australian  Defence  Science  &  Technology  Organisation  (DSTO) 

OIM  (’99) 
OIAM  (’05) 

Joint  Forces  Cmd  (JFCOM)  Joint  Forces  Program  Office  (JFPO) 

Stoplight  (’02) 

Old  Dominion  Univ.  Virginia  Modeling  Analysis  &  Simulation 
Center  (VMASC) 

LCI  (’03) 
LCIM  (’03) 

North  Atlantic  Treaty  Organization  (NATO) 

NMI  (‘03) 

DoD  Command  and  Control  Research  Program 

NCW  (’03) 

Carnegie  Mellon  Software  Engineering  Institute  (CMU-SEI) 

SoSI  (’04) 

Defence  Science  &  Technology  Lab.  (Dstl)  Contractor,  QinetiQ 

NTI  (’04) 

Research  Establishment  for  Applied  Science  (FGAN) 

NID  (’05) 

Air  Force  Institute  of  Technology  (AFIT) 

i-Score  (’07) 

Spectrum  of  Interoperability  Model  (SoIM) 


MAIN  CONTRIBUTION:  Interoperability  can  be  measured  in  levels. 

In  1980,  LaVean  acknowledged  in  the  IEEE  Transactions  on  Communications 
that  inter-system  interoperability  was  poor  because  there  existed  a  “lack  of  a  measure  of 
interoperability  by  which  to  state  goals  for  specific  systems.”  (1980:1449)  To  combat 
this  deficiency,  he  created  a  spectrum  of  interoperability  model  (SoIM).  (Ibid:  1448)  He 
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defined  the  two  most  important  measures  of  interoperability  (technical  possibility  and 
management/control  possibility),  assigned  levels  (Table  30),  and  stated  that  by 
“combining  these  two  measures,  it  is  possible  to  derive  a  spectrum  of  interoperability  that 
permits  cost-versus-benefits  tradeoffs.”  (Ibid)  Recognizing  that  the  level  of 
interoperability  may  be  different  for  each  service  that  pairs  of  systems  provide  to  each 
other,  he  proposed  a  visualization  method,  called  an  interoperability  matrix,  which  lists 
services  on  the  rows  of  the  matrix  and  levels  of  interoperability  on  the  columns.  He 
further  proposed  a  current  view  and  a  “future”  view  of  the  interoperability  matrix  in  order 
to  show  evolution  of  the  systems  over  time.  Thus,  the  purpose  of  SoIM  was  to  provide  a 
simple  tool  for  program  managers  to  assess  current  interoperability  of  their  systems  and 
services,  to  set  goals  for  future  interoperability,  and  to  visualize  the  current  and  future 
states  of  interoperability.  Although  SoIM  was  groundbreaking  and  is  possibly  the  earliest 
method  for  measuring  interoperability,  there  is  no  further  mention  of  his  model  after  its 
original  publication  and  it  is  unknown  whether  or  not  it  was  used  by  program  managers 
to  improve  inter-system  interoperability. 


Table  30  SoIM  levels  of  interoperability 


Level 

Name 

Technical 

Measure 

Management/Control 

Measure 

1 

Separate  Systems 

1 

1 

2 

Shared  Resources 

1 

2 

3 

Gateways 

2 

3 

4 

Multiple  Entry  Points 

2 

4 

5 

Conformable/Compatible  Systems 

3 

4 

6 

Completely  Interoperable  Systems 

3 

5 

7 

Same  System 

4 

6 
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Quantification  of  Interoperability  Methodology  (QoIM) 


MAIN  CONTRIBUTION:  Interoperability  is  eorrelated  to  measures  of  effectiveness. 

In  1989,  Mensh,  Kite,  and  Darby  published  a  method  in  the  Naval  Engineer’s 
Journal  called  “The  Quantification  of  Interoperability”  (QoIM)  Working  for  MITRE 
Corp.,  they  may  have  laid  some  of  the  groundwork  for  the  well-known  LISI  model 
published  by  MITRE  nine  years  later  although  they  were  never  credited.  Mensh,  et.  al.’s 
approach  to  interoperability  measurement  is  unique  because  they  associated 
interoperability  with  measures  of  effectiveness  (MOE).  Their  goal  was  to  assess 
interoperability  issues  for  three  mission  areas:  wide  area  surveillance  (WAS),  over-the- 
horizon  targeting  (OTH-T),  and  electronic  warfare  (EW)  by  quantifying  seven 
interoperability  components.  (Mensh,  et.  al.,  1989)  They  stated  that  “interoperability  of 
systems,  units,  or  forces  can  be  factored  into  a  set  of  components  that  can  quantify 
interoperability”  (Ibid:251)  and  identified  the  seven  components  as  media,  languages, 
standards,  requirements,  environment,  procedures,  and  human  factors.  They  specified  an 
arbitrary  MOE  logic  function  for  each  component  and  used  that  logic  function  to  create  a 
truth  table  populated  via  discrete  event  simulation.  For  example,  the  MOE  logic  function 
for  the  Language  component  was  defined  as  “Message  Correctness  =  Intelligibility  and 
Manual  Intervention  &  Error.”  (Ibid:255-256)  The  truth  table  listed  the  binary  MOE 
value  (e.g..  Message  Correctness,  Intelligibility,  and  Manual  Intervention  and  Error)  for 
various  “significant  events”  which  occurred  during  an  exercise  or  simulation — the 
presence  of  zeros  indicated  lack  of  interoperability  during  certain  component  events  and 
the  presence  of  ones  indicated  that  some  level  of  interoperation  occurred.  (Ibid:254-255) 
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A  final  Interoperability  Data  Table  was  formed  showing  the  truth  table  results  for  all 
seven  interoperability  components,  which  “illustrates  the  overall  quantification  of 
interoperability,”  and  “for  specific  events  it  enables  an  evaluation  of  the  interoperability 
of . . .  systems  in  terms  of  the  seven  interoperability  components . . .  [and] 
corresponding. .  .events,”  and  finally,  “having  this  type  of  table  for  two 
different. .  .architectures  enables  a  comparison  of  the  relative  goodness  of  each 
architecture.”  (Ibid:259)  Although  Mensh,  et.  al.  state  that  their  “methodology  for 
quantifying  interoperability  is  being  pursued,”  they  admit  that  “additional  exercises  will 
be  required  and  are  currently  in  the  planning  stages.”  (Ibid)  Aside  from  one  citation  by 
Leite  in  1998  (and  revised  paper  in  2003),  there  are  no  further  mentions  or  apparent  use 
of  this  model  beyond  the  original  journal  in  which  it  was  published. 

Military  Communications  and  Information  Systems  Interoperability  (MCISI) 

MAIN  CONTRIBUTION:  The  distance  between  systems  modeled  as  points  in  space 
indicates  their  interoperability. 

In  1996,  Amanowicz  &  Gajewski  published  an  interoperability  measurement 
model  (MCISI)  designed  to  model  communications  and  information  systems  (CIS) 
interoperability  mathematically.  Noting  that  interoperability  modeling  combines 
operational  requirements,  CIS  data,  standards,  interfaces  and  modeling  facilities,  they  use 
a  colored  cube  to  visualize  their  model  in  which  one  axis  is  level  of  command,  the  second 
is  CIS  services,  and  the  third  is  transmission  medium.  (Amanowicz  &  Gajewski,  1996) 
The  color  of  the  intersections  is  red,  yellow,  or  green  representing  none,  partial,  or  full 
interoperability  of  a  specific  service  through  a  specific  medium  at  a  specified  level  of 
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command.  Amanowicz  &  Gajewski  further  describe  a  set  of  systems  as  points  in  multi¬ 
dimensional  space  with  features  of  these  systems  as  the  coordinates  of  the  points. 
Defining  a  normalized  “distance”  between  two  points  as  d[A,B) ,  they  state  that  when 

d[A,B)  =  0  systems  A  and  B  achieve  full  interoperability  and  when  d[A,B)>\,  the  system 

pair’s  interoperability  decreases.  (Ibid)  They  accommodate  a  set  of  systems  by  creating 
dendrite  (a  broken  line  which  connects  all  points  of  a  set)  arrangements  of  the  systems 
and  state  that  the  best  arrangement  is  the  one  with  the  shortest  dendrite  length.  MCISI 
was  not  institutionalized  after  its  publication. 

Levels  of  Information  System  Interoperability  (LISI)  Model 

MAIN  CONTRIBUTION:  Interoperability  attributes  of  information  systems. 

LISI  is  the  most  prominent  interoperability  measurement  model  within  the 
Department  of  Defense.  It  began  development  at  the  MITRE  Corporation  in  1993  and 
was  published  in  1998  by  the  C4ISR  Architecture  Working  Group  (AWG)  co-chaired  by 
the  Joint  Staff  J6I  and  the  Director,  Architectures  Directorate  of  the  C4ISR  Integration 
Support  Activity  (CIS A)  under  the  direction  of  OSD(ASD(C3I)).  (DoD,  1998)  The  LISI 
report  stated,  “We  lack  a  practical  assessment  process  for  determining  the  interoperability 
maturity  level  or  ‘metric’  of  a  given  system  or  system  pair. .  .The  LISI  Assessment 
Process,  with  its  associated  tool,  system  profiles,  and  data  repository,  fills  these  needs.” 
(Ibid:  ES-7)  LISI  is  a  system  focused  vice  mission  focused  method  applicable  only  to 
information  systems. 

While  CJCSI  62 12.0 1C,  Interoperability  and  Supportability  of  Information 
Technology  and  National  Security  Systems,  (2003)  required  program  managers  to  ensure 
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that  they  complied  with  LISI  requirements,  that  mandate  expired  in  the  latest  version, 
CJCSI  62 12.0 ID.  (2006)  Although  LISI  was  originally  to  be  institutionalized  in  DoDD 
4630.5,  DoDI  4830.8,  DoDI  5000.2,  and  DoDD  5000.1,  this  was  never  accomplished. 
(DoD,  1998)  The  Joint  Staff  maintained  a  repository  of  LISI  profiles  for  acquisition 
programs  for  several  years.  (CJCSI  62 1 2.0 1 C,  2003) 

Like  SoIM,  LISI  describes  levels  of  interoperability  called  maturity  levels. 
Whereas  SoIM  has  seven  levels,  LISI  has  five — Level  0  (Isolated),  Level  1  (Connected), 
Level  2  (Functional),  Level  3  (Domain),  and  Level  4  (Enterprise).  However,  LISI 
improves  upon  SoIM  by  giving  four  attributes  of  the  levels  described  by  the  acronym 
PAID — Procedures,  Applications,  Infrastructure,  and  Data.  The  LISI  Reference  Model  is 
shown  in  Figure  3.  A  web-based  questionnaire  is  completed  in  order  to  generate  the 
Interoperability  Profile  which  contains  information  about  a  system  for  all  four 
interoperability  attributes.  From  the  profile,  an  Interoperability  Metric  can  be  obtained 
which  is  a  triplet  of  metric  type  (Generic,  Expected,  &  Specific),  Level  (0. .  .4),  and  Sub- 
level  (a. .  .z).  The  metric  describes  the  level  of  interoperability  for  one  system  (generic) 
or  a  pair  of  systems  (expected  and  specific).  The  generic  metric  is  the  best  level  of 
interoperability  a  single  system  is  capable  of  whereas  the  expected  metric  describes  the 
highest  common  level  of  interoperability  for  a  system  pair.  The  specific  metric  describes 
the  highest  common  level  of  interoperability  between  two  information  systems  across  all 
PAID  attributes.  LISI  has  been  reviewed  and  critiqued  by  many  other  researchers  since 
its  publication.  Recent  reviews  have  been  written  by  Brownsword,  et  al.  (2004),  Carney 
&  Obemdorf  (2004),  Clark  &  Jones  (1999),  Clark  &  Moon  (2001),  Kasunic  &  Anderson 
(2004),  Morris,  et  al.  (2004),  and  Tolk  (2003),  among  others. 
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Interoperability  Assessment  Methodology  (lAM) 


MAIN  CONTRIBUTION:  Attributes  of  interoperability. 

Leite’s  Interoperability  Assessment  Methodology  was  initially  published  in  the 
pProceedings  of  the  66*  Military  Operations  Research  Society  (MORS)  Symposium 
three  months  after  LISI  was  published  and  was  revised  again  in  1999  and  2003.  It  is 
unknown  if  the  author  was  aware  of  the  LISI  effort,  however  he  did  reference  Mensh,  et. 
al.’s  QoIM  in  his  paper.  Like  QoIM,  lAM  is  based  upon  the  idea  of  “measurement  and 
quantification  of  a  set  of  interoperability  system  components.”  (Leite,  2003: 1)  lAM 
identified  nine  components  (vice  QoIM’s  seven)  which  are  requirements,  standards,  data 
elements,  node  connectivity,  protocols,  information  flow,  latency,  interpretation,  and 
information  utilization.  Each  of  the  nine  components  has  either  a  “yes/no”  answer  or  a 
mathematical  equation  associated  with  it.  Leite  also  defines  “degrees  of  interconnection” 
which  are  connectivity,  availability,  interpretation,  understanding,  utility,  execution,  and 
feedback.  (Ibid:3-8)  He  summarizes  lAM  in  the  form  of  a  flowchart  and  applies  the 
process  to  the  Navy’s  Tactical  Ballistic  Missile  Defense  Program  as  an  example.  His 
methodology  was  not  institutionalized,  but  was  referenced  by  Kasunic  and  Anderson  in 
2004  who  state  that  lAM’s  quality  attributes  can  be  used  to  extend  the  LISI  model  at  the 
mission  slice  level.  (Kasunic  &  Anderson,  2004) 

Organisational  Interoperability  Maturity  Model  for  C2  (OIM) 

MAIN  CONTRIBUTION:  Organizations  interoperate,  but  have  different  interoperability 
attributes  than  technical  systems. 
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In  1998,  the  Australian  Defense  Seience  and  Technology  Organisation  (DSTO) 
completed  a  Command  and  Control  Support  (C2S)  study  in  which  they  described  five 
layers  of  C2  Support  (Telecommunications,  Info  Technology,  Info  Management,  C2 
Process,  and  C2  Framework).  (Clark  &  Jones,  1999)  In  this  study,  they  pointed  out  that 
LISI)  is  strongly  technological,  2)  focuses  on  system  and  technical  compatibility,  and  3) 
does  not  address  higher  layers  of  C2  support.  As  a  result,  Clark  &  Jones  determined  to 
create  an  organizational  extension  to  LISI.  The  result  of  their  labors  is  the  Organisational 
Interoperability  Maturity  Model  (OIM)  first  introduced  in  June  1999  at  the  International 
Command  and  Control  Research  and  Technology  Symposium  (ICCRTS),  then  revised  in 
2003  at  the  same  conference  by  Fewell  and  Clark. 

OIM  was  used  to  “identify  problems  and  evaluate  interoperability  in  a  coalition 
operation”  (Fewell  &  Clark,  2003:3)  Like  LISI,  OIM  defined  five  levels  of 
interoperability  (independent,  cooperative,  collaborative,  combined,  and  unified). 
However,  unlike  LISFs  technically-associated  PAID  attributes,  OIM  defined  four 
attributes  of  organizational  interoperability — 1)  preparation,  2)  understanding,  3) 
command  and  coordination,  and  4)  ethos  (Socio-Cultural  factors).  Fewell  &  Clark 
supplied  detailed  descriptions  of  the  attributes,  identified  multiple  sub-attributes  for  each 
of  the  four  main  attributes  and  used  the  revised  model  to  analyze  the  operational 
interoperability  of  three  scenarios:  1)  the  multi-national  force  participating  in  the 
Australian  led,  1999-2000  International  Force  East  Timor  (INTERFET)  operation,  2)  an 
Australia-US  interoperability  review,  and  3)  the  Multinational  Limited  Objective 
Experiment  2  (MNLOE2)  held  in  February  2003.  OIM  was  reviewed  by  several 
researchers  after  its  initial  introduction  in  1999.  Some  examples  are:  Briscombe,  et  al. 
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(2006),  Brownsword,  et  al.  (2004),  Clark  &  Jones  (1999),  Clark  &  Moon  (2001),  Fewell, 
et  al.  (2004),  Kasunic  &  Anderson  (2004),  and  Morris,  et  al.  (2004)  Dekker  published  a 
modification  to  the  model  which  he  uses  to  analyze  the  Black  Hawk  Down  incident  in 
Mogadishu  in  1993.  (2005)  It  is  unknown  whether  the  OIM  model  has  been 
institutionalized  by  the  Australian  Department  of  Defence. 

Stoplight 

MAIN  CONTRIBUTION:  Operations  &  acquisition  have  interoperability  requirements. 

In  2002,  Hamilton,  et.  al.  published  a  very  uncomplicated  interoperability 
measurement  model  which  they  simply  called  a  Stoplight  model.  They  stated  that 
“interoperability  is  notoriously  difficult  to  measure,”  yet  gave  a  “simplified  model”  to 
measure  it.  (Ibid:20-21)  The  model’s  purpose  is  to  help  decision  makers  understand 
whether  or  not  their  legacy  systems  meet  operational  and  acquisition  interoperability 
requirements  and  is  designed  as  a  two-dimensional  matrix  in  which  “meets  operational 
requirements  (yes/no)”  appears  on  the  rows  of  the  matrix  and  “meets  acquisition 
requirements  (yes/no)”  appears  on  the  columns.  The  intersections  of  the  matrix  are 
colored  red,  yellow,  orange,  and  green  depending  on  how  well  the  specific  type  of 
requirement  is  met.  Hamilton,  et.  al.  give  an  example  of  how  the  color  codings  can  be 
overlaid  on  a  timeline  to  show  the  plan  to  achieve  improved  interoperability  in  the  future. 
This  model  has  not  been  institutionalized  within  the  Department  of  Defense. 

Levels  of  Conceptual  Interoperability  Model  (LCIM) 

MAIN  CONTRIBUTION:  Conceptual  interoperability  bridges  system  interoperability. 
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The  Levels  of  Coneeptual  Interoperability  Model  (LCIM)  was  published  by  Tolk 
&  Muguira  in  2003  with  the  intent  that  it  be  used  to  “beeome  a  bridge  between  the 
eonceptual  design  and  the  teehnieal  design  for  implementation,  integration,  or 
federation,”  and  that  it  be  used  to  “enhanee  the. .  .DoD  Net-Centrie  Data  Strategy  for  the 
Global  Information  Grid  (GIG).”  (Tolk  &  Muguira,  2003:1)  Additionally,  they  state  that 
it  can  be  used  as  a  framework  “to  determine  in  the  early  stages  of  the  federation 
development  process  whether  meaningful  interoperability  between  systems  is  possible.” 
(Ibid)  LCIM  focuses  on  the  world  of  modeling  and  simulation,  and  initially  gave  five 
levels  of  interoperability  but  later  extended  to  seven  as  a  result  of  “new  research  at 
VMASC  and  as  the  response  to  critique  by  the  scientific  community.”  (Tumitsa  &  Tolk, 
2006:1)  The  final  levels  are  Level  0 — No  interoperability,  Level  1 — Technical 
interoperability.  Level  2 — Syntactic  Interoperability,  Level  3 — Semantic  Interoperability, 
Level  4 — Pragmatic  Interoperability,  Level  5 — Dynamic  Interoperability,  and  Level  6 — 
Conceptual  Interoperability.  LCIM  has  traction  within  the  modeling  and  simulation 
community.  LCIM  reviewers  include  Brownsword,  et  al.  (2004),  Kasunic  &  Anderson 
(2004),  and  Morris,  et  al.  (2004). 

Layers  of  Coalition  Interoperability  (LCI) 

MAIN  CONTRIBUTION:  Operational  interoperability  is  an  extension  of  technical 
interoperability. 

Also  in  2003,  Tolk  introduced  a  different,  but  similarly  acronymed,  Layers  of 
Coalition  Interoperability  (LCI)  model  which  defines  nine  layers  of  interoperability.  He 
shows  that  there  is  a  continuum  between  technical  interoperability  and  operational 
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interoperability  rather  than  a  distinct  breakpoint  between  the  two  and  that  the  interface 
between  technical  and  operational  interoperability  is  made  at  the  knowledge/awareness 
layer.  The  nine  layers  in  LCI  are,  from  lowest  to  highest,  1)  Physical  Interoperability,  2) 
Protocol  Interoperability,  3)  Data/Object  Model  Interoperability,  4)  Information 
Interoperability,  5)  Knowledge/ Awareness,  6)  Aligned  Procedures,  7)  Aligned 
Operations,  8)  Harmonized/Strategy  Doctrines,  and  9)  Political  Objectives.  These  layers 
are  framed  by  a  “common  model  of  the  operation.”  Tolk  proposes  possible  metrics  for 
his  model  as  those  contained  in  the  NATO  Code  of  Best  Practice  for  C2  Assessment 
(Stenbit,  et.  ah,  2002a),  Code  of  Best  Practice  for  Experimentation  (Stenbit,  et.  ah, 
2002b),  and  Network  Centric  Warfare  Metrics  Framework  (Alberts,  et.  ah,  2000)  LISI 
and  NMI  were  referenced  by  Tolk,  but  OIM  was  not.  Tolk  claims  that  LCI  is  not  meant 
to  be  a  “universal  replacement”  for  other  frameworks,  but  is  meant  to  be  used  to  “help 
formulate  layered  models.”  (Tolk,  2003:17)  LCI  has  been  cited  and  briefly  reviewed  by 
Morris,  et  al.  (2004). 

NATO  C3  Technical  Architecture  Reference  Model  for  Interoperability  (NMI) 

MAIN  CONTRIBUTION:  Same  as  LISI. 

Version  four  of  this  NATO  reference  model  was  published  in  March  2003  and 
according  to  Morris,  et  ah,  it  was  updated  to  closely  reflect  the  LISI  model  in  December 
2003.  (2004)  It  is  no  longer  available  on  the  NATO  website.  NMI  originally  described 
four  degrees  of  interoperability  (not  including  degree  0  which  was  no  interoperability). 
The  four  degrees  mapped  directly  to  LISFs  top  four  levels  and  were  given  as:  1) 
unstructured  data  exchange,  2)  structured  data  exchange,  3)  seamless  sharing  of  data,  and 
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4)  seamless  sharing  of  information  which.  NMI  was  reviewed  by  Brownsword,  et  al. 
(2004),  Kasunic  &  Anderson  (2004),  Morris,  et  al.  (2004),  Tolk  &  Muguira  (2003),  and 
Tolk  (2003). 


Net  Centric  Warfare  Maturity  Model  (NCW) 

MAIN  CONTRIBUTION:  Interoperability  occurs  in  physical,  information,  cognitive, 
and  social  domains;  lack  of  interoperability  increases  difficulty  in  accomplishing  the 
mission. 

Alberts  &  Hayes  published  Power  to  the  Edge  in  2003  and  included  a  Net  Centric 
Warfare  Maturity  Model  (NCW)  in  their  chapter  on  interoperability.  Besides 
emphasizing  the  need  for  interoperability  in  military  operations,  they  point  out  that 
interoperability  underpins  the  tenents  of  net-centric  warfare.  (Alberts  &  Hayes,  2003) 
Specifically,  they  state  that  interoperability  must  be  present  in  four  domains:  physical, 
information,  cognitive,  and  social.  They  correlate  interoperability  to  mission 
effectiveness  by  stating  that  “a  lack  of. .  .interoperability  on  the  part  of  an  entity. .  .makes 
it  difficult  for  them  to  contribute  to  the  mission.”  (Ibid:  108)  NCW  models  the  maturity 
of  situational  awareness  and  command  and  control  in  the  context  of  interoperability 
levels  (Table  31).  The  five  interoperability  levels  in  the  model  are  defined  as  Level  0  - 
limited  interoperability.  Level  1 — more  entities  share  information,  Level  2 — 
collaborative  environments  and  processes,  Level  3 — shared  awareness  in  the  information 
and  cognitive  domains,  and  Level  4 — interoperability  in  the  social  domain. 
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Table  3 1  NCW  maturity  model  (Alberts  &  Hayes,  2003) 


Command  and  Control 

Traditional 

Collaboration 

Self¬ 

synchronization 

Developing 

Situational 

Awareness 

Shared 

Awareness 

3 

4 

Information 

Sharing 

1 

2 

Organic 

Sources 

0 

System-of-Systems  Interoperability  (SoSI)  Model 

MAIN  CONTRIBUTION:  System-of-system  interoperability  researeh  is  founded  upon 
operational,  eonceptual,  and  programmatic  interoperability. 

This  simple  model  was  published  in  2004  by  the  Camegie-Mellon  University 
Software  Engineering  Institute  (CMU-SEI)  and  was  developed  to  facilitate  system-of- 
systems  interoperability  research.  (Morris,  et  ah,  2004)  SoSI  is  founded  upon  three  types 
of  interoperability  (operational,  constructional,  and  programmatic)  and  the  activities 
associated  with  each.  While  it  is  a  useful  way  of  developing  and  integrating  systems-of- 
systems,  SoSI  lacks  metrics  to  specifically  measure  interoperability,  however  it  provides 
a  framework  in  which  an  analyst  can  use  his/her  own  metrics.  The  SoSI  report  also 
summarizes  EISI,  OIM,  NMI,  ECIM,  and  LCI.  SoSI  has  not  been  institutionalized  within 
the  Department  of  Defense. 


Non-Technical  Interoperability  (NTI)  Framework 

MAIN  CONTRIBUTION:  Social,  personnel,  and  process  interoperability,  as  well  as 
organizational  interoperability,  are  valid  types  of  non-technical  interoperability. 

Stewart,  et  ah,  introduced  the  Non-Technical  Interoperability  (NTI)  framework  in 
2004  to  allow  the  United  Kingdom’s  (UK)  Ministry  of  Defence  (MOD)  “to  understand 
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these  aspects  of  interoperability  better  and  to  mitigate  potential  frictional  factors  in 
multinational  forces.”  They  felt  that  OIM  was  a  “useful  top-level  framework”  for  the 
data  they  captured  in  their  own  research,  but  recognized  that  it  did  not  cover  social, 
personnel,  and  process  interoperability.  The  four  enabling  OIM  attributes  form  the  core 
of  the  NTI  framework  which  provides  a  more  detailed  breakdown  of  these  attributes. 
While  a  complete  set  of  metrics  was  not  provided  by  Stewart,  et  ah,  they  did  propose  a 
Multinational  Forces  Cooperability  Index  which  provides  a  score  of  1,  2,  4,  8,  12,  or  16 
for  two  (preparedness  and  understanding)  of  the  four  attributes.  The  NTI  framework  was 
developed  as  result  of  45  interviews  with  UK  military  officers  ranging  in  rank  from  Army 
Captain  to  3-star  General.  It  is  unknown  if  NTI  has  been  institutionalized  within  the  UK 
Ministry  of  Defence. 

Revised  NATO  Interoperability  Directive  (NID) 

MAIN  CONTRIBUTION:  Levels  of  interoperability  can  be  given  in  linguistic  terms. 

Schade  notes  that  LISI  takes  a  system  view  of  interoperability  and  NCW  takes  a 
force  view.  (2005)  He  points  out  that  the  NATO  Interoperability  Directive  also  uses  a 
system  view  of  interoperability,  but  documents  poorly  labeled  levels  of  interoperability. 
He  updates  the  NID  labeling  scheme,  by  applying  linguistic  terminology  extracted  from 
Alberts  &  Hayes  (2003),  with  the  following.  Level  0 —  missing  interoperability.  Level 
1 — physical  interoperability.  Level  2 — syntactic  interoperability.  Level  3 — semantic 
interoperability.  Level  4 — pragmatic  interoperability. 
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Organisational  Interoperability  Agility  Model  (OIAM) 


MAIN  CONTRIBUTION:  There  are  levels  of  ability  of  organizations  to  be  agile  in  their 
interoperation. 

Kingston,  et.  al.  of  the  Australian  Defence  Science  and  Technology  organization 
(DSTO)  published  the  Organisational  Interoperability  Agility  Model  (OIAM)  in  2005.  It 
“builds  on  the  Organizational  Interoperability  Model  developed  by  Clark  and  Jones”  and 
“aims  to  capture  the  dynamic  aspects  of  working  in  coalitions  including  the  ability  of  an 
organization  to  contribute  to  the  rapid  formation  and  reformation  of  coalitions,  including 
novel  ones.”  (Kingston,  et.  al.,  2005:2)  Organizational  agility  is  defined  as  “a  single 
organization’s  potential  to  have  agile  interfaces  to  other  organizations  in  future  coalition 
operations”  and  “assesses  an  organization’s  ability  to  adapt  to  changing  circumstances.” 
(Ibid:3)  Aligning  with  OIM,  OIAM  uses  five  levels  of  organizational  agility  (Static, 
Amenable,  Accommodating,  Open,  and  Dynamic)  as  well  as  the  four  OIM  attributes, 
combining  preparation  and  understanding.  The  model’s  developers  state  they  are  at  the 
beginning  of  their  research  on  organizational  agility  and  that  they  plan  to  develop 
additional  metrics  and  perform  case  studies  in  order  to  refine  the  model.  As  a  new 
model,  it  has  not  yet  been  institutionalized  by  the  Australian  Department  of  Defence. 

The  Layered  Interoperability  Score  (i-Score) 

MAIN  CONTRIBUTION:  Interoperability  measurements  are  operational  process  specific 
and  have  a  maximum  value. 

The  Layered  Interoperability  Score  (i-Score)  is  a  quantitative  method  of 
measuring  the  interoperability  of  all  types  of  systems  in  the  context  of  an  operational 
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process.  (Ford,  et.  al.,  2007a;  2008b)  It  makes  use  of  existing  arehitecture  data  and 
accommodates  more  than  one  type  of  interoperability.  Unique  to  the  i-Seore  method  is  a 
means  of  determining  a  realistie  upper  limit  on  interoperability  for  the  systems  supporting 
the  operational  proeess.  The  i-Score  method  accommodates  custom  layers  which  allow 
the  analyst  to  compensate  the  i-Seore  measurement  for  any  number  of  interoperability- 
related  performanee  factors  such  as  bandwidth,  protocols,  mission  capability  rate, 
probability  of  conneetion,  or  atmospherie  effects,  among  others.  Also  possible  are  cost, 
schedule,  reliability,  and  performance  layers  to  measure  the  impact  of  various 
programmatic  changes  on  the  interoperability  of  the  process.  The  method  can  be  used  to 
make  non-traditional  interoperability  measurements  such  as  organizational  or  policy 
interoperability  measurements.  The  i-Score  method  has  not  been  institutionalized  within 
the  Department  of  Defense. 
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A4.  Summary  of  Extant  Interoperability  Measure  Formats 
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A5.  A  Survey  of  System  Taxonomies 


System  Science  System  Taxonomies 

Ludwig  von  Bertalanffy,  a  biologist  accepted  by  many  as  the  father  of  general 
systems  theory  (GST),  noted  that  systems  are  everywhere.  (1955)  He  defined  them  as 
“complexes  of  elements  standing  in  interaction,”  and  promptly  classified  them  as  open  or 
closed,  a  classification  which  certainly  originated  long  before  von  Bertalanffy’s  time  in 
the  disciplines  of  physics,  chemistry,  and  biology. 

Kenneth  Boulding,  an  economist  and  the  first  president  of  the  Society  for  General 
Systems  Research  (now  known  as  the  International  Society  for  the  Systems  Sciences)  and 
cofounder  of  GST,  published  a  creative  and  more  detailed  classification  of  systems  which 
hierarchically  classifies  systems  as  1)  frameworks,  2)  clockworks,  3)  thermostats,  4) 
cells,  5)  plants,  6)  animals,  7)  human  beings,  8)  social  organizations,  and  9) 
transcendental  systems.  (1956)  Boulding’s  classification  scheme  was  self-described  both 
as  a  hierarchy  of  complexity  and  as  a  systematic  framework  in  which  he  referred  to  each 
of  the  nine  classifications  in  the  hierarchy  as  a  level.  Approximately  thirty  years  later, 
Boulding  proposed  a  new,  but  related  classification,  stating  that  systems  were  either  static 
or  dynamic  and  that  “something  of  a  hierarchy”  of  all  systems  which  “correspond  to 
something  in  the  real  world”  included  systems  which  were  either  mechanical,  cybernetic, 
positive-feedback,  creodic,  reproductive,  demographic,  ecological,  evolutionary,  human, 
and  social.  (1985:  18)  He  then  agglomerates  these  ten  as  physical,  biological,  or  social 
systems  and  organizes  the  remainder  of  his  book  around  discussions  of  the  world  as  not 
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just  a  physical,  biological,  and  social  system,  but  also  as  an  economic,  political, 
communication,  and  evaluative  system. 

Twelve  years  after  Boulding  published  his  first  classification  scheme,  Jordan 
published  a  taxonomy  of  systems  which  grouped  systems  according  to  “intuitive  guesses” 
of  three  “organizing  principles”  each  holding  two  “polar  opposite”  properties.  (1968:  44) 
Jordan  defined  eight  cells  (classifications)  in  his  taxonomy,  which  were  derived  from  his 
principle-property  framework,  by  taking  one  property  from  each  of  the  three  principles — 
1)  rate  of  change  (structural/static  or  fiinctional/dynamic),  2)  purpose  (purposive,  non- 
purposive),  and  3)  connectivity  (mechanistic/organismic).  As  an  example,  Checkland 
uses  the  taxonomy  to  classify  a  road  network  as  a  structural,  purposive,  mechanically- 
connected  system  but  a  mountain  range  as  a  structural,  non-purposive,  mechanically 
connected  set  of  entities.  (1981)  Checkland  uses  this  logic  to  critique  Jordan,  noting  that 
in  his  belief,  Jordan  erroneously  “ascribes  the  purpose,  or  lack  of  it,  to  the  system  itself’ 
rather  than  to  the  system’s  creator.  (Ibid:  108) 

Thus,  Checkland  takes  Jordan’s  taxonomy  as  a  foundation,  merges  some  ideas 
from  Boulding  and  creates  what  he  calls  a  systems  typology  which  includes  five  classes 
of  systems  (natural,  design  physical,  design  abstract,  human  activity,  and  transcendental 
systems).  The  purpose  of  his  typology  is  to  identify  classes  of  entities  based  upon  their 
origin.  According  to  Checkland’ s  typology,  the  set  of  natural  systems,  which  includes 
both  types  of  designed  systems  as  well  as  the  human  activity  systems,  and  the  set  of 
transcendental  systems  are  disjoint  sets.  Checkland  is  quite  confident  in  the 
completeness  of  his  typology  and  declares  that  it  “completes  a  simple  systems  map  of  the 
universe  which,  as  far  as  system  classes  is  (sic)  concerned,  is  itself  complete.”  (Ibid,  III) 
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Wilson,  a  colleague  of  Cheekland,  adopted  a  revision  of  Cheekland’s  typology, 
calling  it  a  system  elassifieation  instead.  (1990)  He  removed  transeendental  systems 
from  the  elassifieation  and  restated  the  four  remaining  classes  of  systems  as  natural, 
designed,  human  activity,  and  social  and  cultural  systems.  Wilson  created  his 
elassifieation  of  systems  in  order  to  help  refine  the  definition  of  the  word  system  to  a 
level  that  would  be  useful  in  modeling. 

Probably  without  prior  intention,  Aekoff,  in  his  oft-cited  “Toward  a  System  of 
Systems  Coneepts,”  published  a  system  taxonomy  of  sorts,  formed  by  definitions  of 
various  types  of  systems.  (1971)  While  definitely  not  hierarehical  nor  mutually 
exclusive,  his  list  of  system  types  is  never-the-less  useful.  Aekoff  defines  abstract, 
concrete,  elosed,  open,  static  (one-state),  dynamie  (multi-state),  homeostatic,  state- 
maintaining,  goal-seeking,  multi-goal  seeking,  purposive,  purposeful,  ideal-seeking, 
variety-increasing,  and  variety-deereasing  systems.  Without  explieitly  stating  so,  he 
infers  that  other  types  exist,  but  states  that  he  defined  “the  most  important  types  of 
systems.”  (Ibid:  661) 

Valdma  recently  published  a  elassifieation  seheme  for  information,  but  noted  that 
an  analogous  seheme  exists  for  elassifying  systems.  (2007)  His  four-level,  hierarchieal 
elassifieation  of  systems  directly  mirrors  his  information  classification  model  and  puts 
deterministie  systems  at  the  lowest  level,  followed  by  probabilistic  systems,  then 
uncertain  systems  (sub-grouped  into  uneertain-deterministic,  and  uncertain-probabilistie), 
and  finally,  fiizzy  systems  (sub-grouped  into  fuzzy-deterministic  and  fuzzy  probabilistic). 
His  stated  purpose  in  creating  the  elassifieation  is  as  a  “first  step  in  studying  the  non- 
deterministie  phenomena”  in  the  universe.  (Ibid,  265) 
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Systems  Engineering  System  Taxonomies 


In  1957,  Goode  and  Machol  wrote  in  System  Engineering,  that  systems  should  be 
classified  “on  the  basis  of  the  types  of  inputs  with  which  they  must  cope.”  (1957:  299) 
They  further  defined  this  set  of  inputs  as  1)  input  which  is  always  the  same  or  is  of  many 
types,  2)  input  which  occurs  periodically  (or  very  infrequently),  and  3)  input  which  does 
or  does  not  seek  to  destroy  the  system.  Their  rationale  for  developing  the  classification 
was  to  aid  in  the  definition  of  steps  to  be  followed  in  order  to  find  the  “solution  of  the 
problem  of  a  large-scale  or  complex  system.”  (Ibid:  302) 

Hall’s  A  Methodology  for  Systems  Engineering,  published  five  years  later,  has  no 
direct  reference  to  system  classification,  but  indirectly  describes  a  classification  of  natural 
vice  man-made  systems,  discusses  open  and  closed  systems,  and  references  von 
Bertalanffy’s  property  of  the  hierarchical  order  of  systems,  (1962)  Interestingly,  Hall 
interprets  von  Bertalanffy’s  classification  as  a  method  useful  in  partitioning  systems  into 
subsystems,  loosely  inferring  classification  can  be  used  in  design,  and  also  states  that  a 
system  classification  is  useful  in  “enhancing  the  meaning  of  system.”  (Ibid:  63,  68) 

Martin,  in  his  Systems  Engineering  Guidebook,  indirectly  classifies  systems  by 
classifying  product  types,  relating  them  to  systems  by  stating  that  systems  are  comprised 
of  components,  and  components  are  comprised  of  one  or  more  basic  product  types. 

(1997)  His  basic  product  types,  which  he  correctly  notices  are  not  mutually  exclusive, 
are  hardware,  software,  personnel,  facilities,  data,  materials,  services,  and  techniques. 

His  rationale  for  creating  a  taxonomy  of  product  types  is  to  create  a  checklist  “to  ensure 
that  bases  are  covered”  meaning  that  the  required  behavior  for  a  system  should  not  just  be 
allocated  to  hardware  and  software.  (Ibid:  24) 
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Shenhar,  and  Shenhar  &  Bonen  proposed  a  taxonomy  of  systems  in  order  to 
demonstrate  that  system  engineering  design  and  management  methods,  as  well  as  the 
type  of  system  engineering  culture  and  style,  which  are  appropriate  for  one  type  of 
system  are  inappropriate  for  another.  (1995,  1997)  Their  taxonomy  is  two-dimensional 
and  classifies  systems  “according  to  four  levels  of  technological  uncertainty  (low, 
medium,  high,  and  super-high  tech),  and  three  levels  of  system  scope  (assembly,  system, 
and  array).”  (Ibid:  137)  They  cite  the  space  shuttle  as  an  example  of  a  system  which 
NASA  initially  advertised  to  Congress  as  high-tech  but  making  use  of  existing 
technologies,  but  which,  in  hindsight,  should  have  been  managed  as  a  super-high-tech 
system  making  use  of  many  not  yet  developed  technologies  and  methods.  Shenhar  & 
Bonen  state  that  an  understanding  of  their  taxonomy  and  a  proper  classification  of  the 
space  shuttle  as  a  system  could  possibly  have  prevented  schedule  delays  and  even  might 
have  prevented  the  Challenger  tragedy  as  NASA  would  have  been  “more  keenly  aware  of 
the  possibility  of  trouble.”  (Ibid,  144)  Shenhar  &  Bonen  admit  that  their  framework  “is 
not  conclusive”  and  requires  further  refinements  and  investigation,  but  believe  that  it  is 
useful  in  “finding  better  and  more  effective  ways  to  manage  the  creation  of  different 
kinds  of  systems.”  (Ibid,  145) 

Maier  focused  his  research  on  the  topic  of  architecting  systems-of-systems. 

(1999)  He  argued  that  systems-of-systems  must  possess  “operational  and  managerial 
independence  of  the  systems  components”  and  provided  a  “limited  taxonomy”  in  which 
system-of- systems  are  considered  a  “useful  taxonomic  distinction”  separate  from 
monolithic  systems.  (1999:267-284)  He  further  subdivided  the  taxonomic  grouping  of 
system-of-systems  into  virtual,  voluntary,  and  directed  categories. 
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Kovacic’s  taxonomy  provides  “definition  to  the  variety  of  fields  that  hold  claim  to 
the  term  systems”  and  reduces  the  set  of  systems  into  “meaningful  related  clusters.” 
(2005)  He  erroneously  states  that  his  taxonomy  uniquely  uses  complexity  as  its  basis,  as 
Boulding,  whom  Kovacic  cites,  called  his  taxonomy  “a  hierarchy  of  complexity.” 
(Boulding,  1956:  200),  Kovacic  makes  a  unique  application  of  complexity  theory,  but 
more  importantly  notes  that  a  good  taxonomy  must  be  inclusive,  definitive,  reductive, 
and  applicable.  (2005)  Additionally,  he  notes  that  systems  are  difficult  to  classify 
because  they  are  perceptions  of  the  observer.  Kovacic’s  taxonomy  is  three-dimensional 
with  decomposed/un-decomposed,  complex/simple,  and  loosely  bounded/tightly  bounded 
as  the  three  dichotomous  categorizations. 

Gideon  et  al.,  published  a  taxonomy  of  systems-of-systems  in  order  to  aid  in  the 
understanding  of  the  nature  and  attributes  of  systems-of-systems  and  because  “a  clearly 
defined  classification  scheme  is  essential  in  developing  common  systems  engineering 
architectures  and  methodologies.”  (2005)  While  they  admit  that  their  taxonomy  “may 
not  be  complete  or  even  necessarily  correct,”  it  represents  one  of  the  first  attempts  at  a 
classification  scheme  specifically  for  systems-of-systems.  Their  final  taxonomy 
subordinates  systems-of-systems  to  systems  in  general,  then  defines  sub-classifications  of 
acquisition  type  (dedicated  or  virtual),  operational  type  (chaotic,  collaborative,  or 
directed),  and  domain  type  (social,  conceptual,  or  physical). 

Blanchard  and  Fabrycky  discuss  classification  of  systems  in  Systems  Engineering 
and  Analysis,  but  caveat  by  saying  that  the  classifications  they  included  are  “only  some 
of  those  that  could  be  presented”  and  indicate  that  systems  can  be  classified  “for 
convenience  and  to  provide  insight  into  their  wide  range.”  (2006:  6)  They  take  the  path 
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of  dichotomies  as  Jordan  did  and  defined  systems  as  natural  and  man-made,  physical  and 
conceptual,  static  and  dynamic,  and  closed  and  open.  As  is  the  case  with  others  who 
have  classified  systems,  their  classification  scheme  proposes  agglomerations  which 
overlap — a  property  to  be  expected  and  one  that  is  useful.  Blanchard  and  Fabrycky 
acknowledged  tie-in  between  systems  engineering  and  systems  science  and  partially 
aligned  their  system  classifications  to  the  nine  levels  of  complex  systems  proposed  by 
Boulding. 
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A6.  Operational  Process  Modeling  Methods 

_ Table  32  Operational  process  modeling  methods _ 

ACT  Formalism 

A  Language  for  Process  Specification  (ALPS) 

AP213  Protocol  within  ISO  10303 
UML  2.0  Behavior  Diagram 
EPFL’s  Petri  Net  Representation 
Core  Plan  Representation  (CPR) 

Entity-Relationship  (E-R)  Model 
Functional  Flow  Block  Diagram  (FFBD) 

Gantt  Chart 

Generalized  Activity  Network  (GAN) 

Hierarchical  Task  Networks  (HTN) 

Integration  Definition  for  Function  Modeling  (IDEFO) 

Process  Flow  and  Object  State  Description  Capture  Method  (IDEF3) 

Issues,  Nodes,  Ordering,  Variable,  and  Auxiliary  (<I-N-OVA>)  Constraint  Model 
Knowledge  Interchange  Format  (KIF) 

Open  Planning  Architecture  (O-Plan)  Task  Formalism 
OZONE 

Parts  and  Action  (PAct) 

Product- Activity-Resource  Model  for  Realiz.  of  Electro-Mech.  Assemblies  (PAR2) 
Part  49  of  ISO  10303 

Program  Evaluation  and  Review  Technique  (PERT)  Network 
Petri  Net 

Process  Flow  Representation  (PFR) 

Process  Interchange  Format  (PIF) 

Quirk  Models 

Visual  Process  Modeling  Language  (VPML)  (pre-1998  version) 

Visual  Process  Modeling  Language  (VPML)  (post- 1998  version) 

AND/OR  Graph 

Data  Flow  Diagram  (DFD) 

Digraph 

State  Transition  Diagram  (STD) 

SysML  Activity  Diagram 
Tree  Structure 

Process  Specification  Language  (PSL) 

Flow  Chart 

DoD  Architecture  Framework  (DoDAF)  Functionally  Decomposed  Activity  Diagram 
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A7.  Operational  Effectiveness  Measurement 


Hierarchy  of  Measures  of  Operational  Effectiveness 

A  hierarchy  of  measures  was  first  described  by  the  Military  Operations  Research 
Society  (Sweet,  et.  ah,  1985)  and  was  later  adopted  by  the  Department  of  Defense  with 
minor  modification  (Grimes,  2006).  This  hierarchy  also  seems  to  be  widely  accepted 
outside  these  two  communities,  with  slight  modifications,  and  describes  a  hierarchical  set 
of  measures  ranging  from  dimensional  parameters,  to  measures  of  performance,  to 
measures  of  effectiveness,  to  measures  of  force  effectiveness,  and  finally  to  measures  of 
policy  effectiveness  (Green,  2001;  Stenbit,  et.  ah,  2002).  This  hierarchy  is  often  rendered 
as  a  pyramid  or  onion-skin  model  although  it  shouldn’t  be  viewed  that  rigidly.  Similarly, 
many  authors  strictly  apply  dimensional  parameters  to  objects,  measures  of  performance 
to  systems,  measures  of  effectiveness  to  systems  within  an  environment,  measures  of 
force  effectiveness  to  systems  as  part  of  a  force,  and  measures  of  policy  effectiveness  to 
high-level  policy  decisions,  but  there  should  be  flexibility  in  order  to  accommodate 
different  system  types  (e.g.,  organizational  systems).  The  preferred  term  in  this  research, 
measure  of  operational  effectiveness  (MoOE),  fits  in  the  range  of  measurements  between 
measure  of  performance  and  measure  of  force  effectiveness,  inclusive. 

Operational  Effectiveness  Assessment 

Operational  effectiveness  assessment  occurs  during  operational  planning, 
operational  execution,  and  post-operation  analysis.  This  is  well  documented  in  DoD  joint 
operational  planning  and  joint  operations  publications  (JP  5-0,  2006;  JP  3-0,  2006).  A 
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common  means  of  determining  operational  effectiveness  is  through  measurement  of 
appropriate  factors  related  to  the  operation.  Although  these  measures  can  be  qualitative, 
Keeney  states  that  quantifying  the  measures  “clarifies  the  meaning  of  the  (operational) 
objectives,  and  this  clarity... facilitates  all  aspects  of  decisonmaking.”  (Keeney, 

1992:129). 

JP  3-0  Joint  Operations  (2006)  and  JP  5-0,  Joint  Operational  Planning  (2006) 
define  assessment  as  “the  process  that  measures  progress  of  the  joint  force  toward 
mission  accomplishment”  and  state  that  commanders  “continuously  assess... the  progress 
of  operations,  and  compare  them  to  their  initial  vision  and  intent.”  (Ibid:III-57) 
Operational  assessment  uses  both  measures  of  effectiveness  (MOE)  and  measures  of 
performance  (MoP)  (Ibid:III-59).  Although  JP  3-0  and  JP  5-0  associate  MOEs  with 
strategic  assessment  and  MOPs  with  tactical  assessment,  since  an  operation  in  the  context 
of  this  dissertation  can  be  strategic  or  tactical,  the  generalized  term,  measure  of 
operational  effectiveness  (MoOE),  will  be  used  from  here  onward.  Since  the  two  joint 
publications  state  that  an  MoP  is  used  to  measure  task  performance  and  an  MoE  is  used 
to  “determine  progress  of  an  operation  toward  achieving  objectives,”  (Ibid:III-60)  it  is 
appropriate  to  say  that  an  MoOE,  although  retaining  some  characteristics  of  an  MoP  (e.g., 
measurement  of  level  of  operational  tasks,  or  thread,  completion),  is  more  closely  aligned 
with  the  definition  of  an  MOE,  and  hence  is  appropriately  named. 

The  initial  MOEs  defined  during  operational  planning  are  also  called  success 
criteria  (JP  5-0,  2006:111-27).  Indeed,  MOEs  are  defined  early  (Step  #2,  Mission 
Analysis  of  7  steps)  in  the  operational  planning  process  and  become  “the  basis  for  reports 
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to  senior  commanders  and  civilian  leaders  on  the  progress  of  the  operation”  (Ibid) 
because  they  measure  “the  attainment  of  an  end  state,  achievement  of  an  objective,  or 
creation  of  an  effecf  ’  (JP  3-0,  2006:IV-32;  JP  5-0,  2006:111-60).  In  fact,  according  to 
Murray  in  A  Will  to  Measure,  the  desire  to  measure  quantitatively  is  “irresistible”  to 
modem  society  and  the  armed  forces  use  quantitative  methods  to  explain  actions  to 
leaders,  politicians,  and  the  public  (2001 : 134).  Murray  aptly  states  that  “the 
interpretation  of  the  MOE  frequently  forms  the  stmcture  on  which  senior  leaders  base 
their  orders.”  (Ibid) 

JP  5-0  gives  an  example  of  an  operation  (evacuate  all  US  personnel  from  an 
embassy)  and  two  associated  MOEs  (“are  all  personnel  evacuated?”  and  “have  any  rales 
of  engagement  been  violated?”)  which  highlights  the  fact  that  MOEs  are  operation 
dependent  and  that  there  are  often  more  than  one  MOE  associated  with  the  operation. 
(2006:111-27) 

An  appropriate  MOE  must  be  carefully  selected.  According  to  Murray,  MOEs 
which  “adequately  reflect  and  distill  reality  help  decisionmakers  make  informed  and 
timely  decisions,”  while  “poorly  chosen  measures  have  a  multitude  of  negative  effects” 
(2001:134). 


MoOE  Characteristics 

Multiple  researchers  give  desirable  characteristics  of  MoOEs.  Fourteen 
characteristics  are  consolidated  below  (in  no  special  order)  in  Table  33  and  are  discussed 
individually  in  succeeding  paragraphs. 
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Table  33  MoOE  characteristics 
Characteristic _ 

1)  Relevant 

Also  called  “operational” 

Also  called  “mission-oriented” 

2)  Measurable 

3)  Responsive 

Also  called  “sensitive” 

4)  Resourced 

5)  Understandable 

6)  Discriminatory 

7)  Quantitative 

8)  Realistic 

9)  Objective 

10)  Appropriate 

11)  Inclusive 

12)  Independent 

13)  Valid 

14)  Reliable  (Precision) _ 


Relevant.  An  MOE  must  be  relevant  to  the  operation,  or  in  other  words,  mission- 
oriented.  For  example,  during  the  Kosovo  campaign,  NATO’s  focus  on  counting  the 
number  of  vehicles  and  weapons  destroyed  and  the  number  of  sorties  flown  and  bombs 
dropped  “did  not  provide  a  sense  of  whether  Yugoslavian  leaders  were  ready  to  accede  to 
NATO  demands”  but  instead  validated  “performance  requirements”  of  weapons — an 
irrelevant  indicator  of  operational  effectiveness  of  the  campaign.  (Murray,  2001:134).  A 
positive  change  in  the  MOE  value  should  indicate  greater  operational  success.  Similarly, 
a  negative  change  in  the  value  of  the  MOE  value  should  indicate  a  decline  in  operational 
success. 

It  should  not  be  assumed,  however,  that  an  MOE  must  be  a  direct  (also  called 
natural  by  Keeney  (1992:101)  measure  of  the  success  of  the  operation.  Indeed,  often 
direct  measures  violate  criteria  #2  (measurability),  but  measurable  inferential  (also  called 
indicative,  indirect,  or  constructed)  measures  may  be  available.  For  example,  the  MOE 
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“level  of  the  morale  of  the  enemy”  cannot  be  directly  measured,  but  an  inferential 
measure  such  as  “number  of  attacks  per  month  made  by  the  enemy”  may  be  an  adequate 
estimate  of  enemy  morale  in  that  it  indicates  the  enemy’s  desire  to  continue  the  fight.  If  a 
commander  or  leader  must  resort  to  using  inferential  MOEs,  extra  care  must  be  taken  to 
ensure  that  the  MOE  is  relevant.  For  example,  Secretary  McNamara  used  body  count  as 
an  inferential  measure  of  operational  effectiveness  during  the  Vietnam  war,  but  that 
measure  not  only  did  not  reflect  progress  in  winning  the  war,  but  had  many  unintended 
effects  (Murray,  2001)  such  as  failure  of  commander  integrity,  and  possibly  the 
unnecessary  killing  of  civilians. 

Murray  writes  that  it  can  be  difficult  to  discover  good  MOEs  which  accurately 
reflect  the  positive  and  negative  trends  in  operational  effectiveness  because  the 
“underlying  causal  mechanisms  are  exceedingly  difficult  to  determine”  (Ibid:  138).  He 
accurately  observes  that  rarely  is  one  measure  appropriate  but  that  most  operational 
effectiveness  must  be  measured  from  a  complex  set  of  factors. 

Measurable.  An  MOE  must  be  measurable  (Bomman,  1993;  JP  3-0,  2006;  JP  5- 
0,  2006).  Although  JP  5-0  acknowledges  that  MOEs  can  be  qualitative  or  quantitative, 
quantitative  MOEs  are  preferred  because  they  are  “less  susceptible  to  subjective 
interpretation.”  (2006:111-61)  Qualitative  measures  can  usually  be  quantized.  For 
example,  the  MOE,  “are  all  personnel  evacuated,”  is  a  yes  or  no  question  which  can  be 
rendered  as  a  1  or  0.  Similarly,  the  MOE,  “to  what  level  has  the  enemy’s  ability  to  place 
improvised  explosive  devices  (lED)  been  degraded,”  appears  qualitative,  but  infers  that  a 
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scale  can  be  applied  by  using  the  word  level.  If  a  seale  of  0  to  10  lEDs  per  day  is  used, 
then  the  MOE  becomes  quantitative  and  measurable. 

Responsive  (Sensitive).  An  MOE  must  be  responsive,  or  sensitive,  to  ehanges  in 
the  operation.  (Bomman,  2001;  JP  3-0,  2006;  JP  5-0,  2006)  In  order  to  give  the 
eommander  fidelity  of  understanding  on  how  the  effectiveness  of  the  operation  is 
ehanging,  the  MOE  must  be  sensitive  to  changes  in  operational  effectiveness.  For 
example,  if  the  operational  goal  is  to  assist  demoeratie  revolutionaries  in  overthrowing 
their  country’s  dictatorship,  a  measurable  and  relevant,  yet  insensitive  MOE  could  be, 
“number  of  machines  guns  provided  to  the  revolutionaries  eaeh  month.”  Although 
maehine  guns  may  eventually  help  revolutionaries  overthrow  the  dietatorship,  the  time 
required  for  training  and  planning  may  result  in  delayed  progress  toward  the  goal  of  the 
operation.  In  faet,  more  maehine  guns  than  necessary  may  be  provided  in  successive 
months  as  the  operation’s  commander  attempts  to  aecelerate  the  overthrow  of  the 
dietatorship. 

Resourced.  An  MOE  must  have  the  necessary  resourees  (i.e.,  manpower,  money, 
time,  ete.)  alloeated  for  data  collection,  analysis,  and  reporting  (JP  3-0,  2006;  JP  5-0, 
2006).  If  resourees  are  not  allocated  for  assessment,  it  won’t  matter  how  appropriate  the 
MOEs  are,  because  the  measurements  will  not  be  able  to  be  made,  or  the  measurements 
may  be  incomplete  or  inaccurate.  For  example,  if  a  eommander  desires  to  improve  war 
fighting  efficieney  of  his  unit,  but  does  not  have  enough  people  on  staff  to  dedicate  to 
data  gathering  and  analysis,  the  commander  may  find  that  the  effieiency  analysis 
eventually  provided  is  shallow,  too  narrowly  focused,  or  downright  erroneous. 
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Understandable.  This  characteristic  is  self  explanatory  and  is  also  ealled  simple 
by  Bomman  (2001)  and  Green  &  Johnson  (2002).  An  MOE  must  be  understandable 
(Keeney,  1992;  Campbell,  2004)  not  only  to  the  person  making  the  measurement,  but 
also  to  the  leader  whose  decisions  are  based  upon  the  measurement. 

Discriminatory.  Bomman  states  that  an  MOE  must  be  diseriminatory  in  order  to 
“identify  real  differences  between  alternatives”  (1993:2-3).  This  characteristie  is  related 
to  the  characteristics  of  objectivity  and  independence  deseribed  below. 

Quantitative.  Keeney  writes  that  objectives  are  qualitative  and  attributes  (MOEs) 
are  quantitative  (1992).  IP  3-0  states  that  MOEs  are  qualitative  and  MOPs  are 
quantitative  (2006),  however  JP  5-0  states  that  MOEs  are  either  qualitative  or  quantitative 
(2006).  Bomman  insists  that  MOEs  are  quantitative  (1993). 

Realistic.  Although  Bomman  calls  out  realistic  as  a  separate  eharacteristic  of  a 
desirable  MOE,  “realistie”  is  largely  implied  in  the  more  important  eharacteristics  of 
measurable,  resourced,  and  understandable — all  of  which,  if  missing,  result  in  a  MOE 
whieh  is  not  realistic.  Murray  reminds  that  realistic  measures  which  “adequately  distill 
and  accurately  refleet  reality  help  deeisionmakers  make  informed,  timely  deeisions.” 
(2001:134) 

Objective.  Bomman  states  that  measures  can  be  objeetive  or  subjective,  but  lists 
objectivity  as  a  desirable  charaeteristie.  (1993)  Keeney  mentions  that  a  subjeetive,  or 
qualitative,  “stracture”  ean  be  quantified  using  a  value  model.  (1992)  Keeney’s 
philosophy  is  that  any  qualitative  measure  ean  be  rendered  quantitatively.  This  usually  is 
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accomplished  by  applying  a  scale,  or  other  type  of  model,  to  give  meaning  to  the  values 
of  the  measure.  Non-objeetive  measures  are  often  eonstructed  or  proxy  measures. 

Appropriate.  Bornman  states  that  a  measure  of  effectiveness  should  be 
appropriate,  whieh  he  defines  as  relating  to  “aceeptable  standards  and  analysis 
objectives.”  (1993:2-3) 

Inclusive.  Bornman  also  mentions  that  a  desirable  measure  of  effeetiveness  is 
inelusive,  meaning  it  “refleet(s)  those  standards  required  by  the  analysis  objeetives.” 
(1993:2-3)  No  further  elarification  is  offered  although  a  referenee  is  made  to  a  1985 
Military  Operations  Research  Society  workshop  which  developed  some  (or  possibly  all) 
of  the  eharaeteristics  listed  in  the  Bornman  (Army  TRADOC)  handbook.  Unfortunately, 
no  bibliography  was  ineluded  in  the  handbook,  so  it  is  diffieult  to  aecurately  identify  the 
source  which  Bornman  referenced,  although  it  likely  was  the  1985  doeument  referenced 
by  Green  &  Johnson.  (2002) 

Independent.  Independenee  is  an  important  characteristic  of  an  MOE,  because  it 
drives  the  analyst  to  find  measures  of  operational  effectiveness  whieh  are  not  confounded 
with  eaeh  other.  This  is  desirable  from  a  eommander’s  perspective  since  it  results  in 
measures  which  are  distinet  from  each  other.  This  allows  eommanders  to  change  certain 
aspeets  of  their  operation  and  measure  that  ehange  without  affecting  other  aspeets  of  the 
operation.  In  practice,  it  is  diffieult  to  deseribe  independent  MOEs.  Design  of 
experiments  and  response  surfaee  methodology  theory  reeommend  if  independence  of 
faetors  is  impossible  (i.e.,  due  to  eost,  ease  of  measurement,  or  other  reasons),  then  eare 
should  be  taken  to  ensure  important  factors  are  eonfounded  with  negligible  faetors.  This 
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minimizes  the  impact  due  to  variable  dependence.  This  same  philosophy  can  be  applied 
to  MOE  selection  by  choosing  measures  of  important  effects  which  are  not  confounded 
with  a  measure  of  another  equally  important  effect. 

Measure  independence  was  listed  as  “desired  but  not  essential”  by  Bomman 
(1993)  and  was  listed  as  a  desired  characteristic  of  MOEs  by  Green  &  Johnson.  (2002) 
The  NATO  Code  of  Best  Practice  for  C2  Assessment  states  that  analysis  of  operations  is 
challenging  due  to  “the  number  of  confounded  variables”  (Stenbit,  et.  ah,  2002:191)  and 
and  further  states  that  independent  measures  are  controllable. 

Valid.  Bullock  states  that  a  measurement  is  valid  if  it  “reflects  the. .  .attributes  it 
was  supposed  to  represent.”  (2006:8) 

Reliable.  Finally,  Bullock  writes  that  a  measure  must  be  reliable  (which  he  also 
calls  precision),  meaning  that  the  measurement  process  must  be  able  to  yield  a  consistent 
and  repeatable  measurement. 


MOE  Types  and  Domains 

Keeney  lists  three  types  of  MOEs  (which  he  called  attributes) — natural, 
constructed,  and  proxy.  (1992)  A  natural  MOE  is  one  which  has  “a  common 
interpretation  to  everyone”  such  as  annual  profit  in  millions  of  dollars.  (Ibid:  101)  A 
constructed  MOE  is  not  a  direct  or  natural  measure,  but  includes  a  subjective  judgment. 
(Campbell,  2004)  For  example,  the  Richter  scale  for  measuring  earthquake  intensity  is  a 
constructed  measure.  (Ibid)  Campbell  points  out  that  constructed  measures,  as  they 
become  commonly  known  and  understood  can  become  natural  measures  (e.g.,  the  Dow 
Jones  Industrial  Average  and  the  Gross  National  Product).  Finally,  the  proxy  MOE  is  an 
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inferential,  or  indicator,  measure  which  describes  something  related  to  what  is  actually 
wanted  to  be  measured.  For  example,  the  level  of  Carbon  14  is  an  indicator  of  the  age  of 
an  object.  A  natural  measure  is  often  objective  whereas  constructed  and  proxy  measures 
are  often  subjective.  (Bomman,  1993)  Although  natural  MOEs  are  the  most  desirable, 
more  often  than  not,  constructed  or  proxy  MOEs  1)  more  readily  meet  the  MOE  criteria, 
2)  cost  less  to  gather  data  and  measure,  or  are  3)  more  easily  discovered.  An  MOE  is 
measured  in  the  physical,  information,  or  cognitive  domain  .(Stenbit,  et.  al.,  2002) 
Generally,  measurements  in  the  physical  domain  are  easier  to  make  and  those  in  the 
cognitive  domain  are  more  difficult. 


MOE  Summary 

An  MOE  is  defined  as  “a  standard  used  to  assess  changes  in  the  production  of  a 
desired  operational  effect.”  It  is  exists  within  the  range  of  measure  of  performance, 
measure  of  effectiveness,  and  measure  of  force  effectiveness  in  the  hierarchy  of 
measures.  The  three  types  of  MOE — natural,  constructed,  and  proxy — exist  within  the 
physical,  information,  and  cognitive  domains.  The  best  MOEs  are  relevant,  measurable, 
responsive,  resourced,  understandable,  discriminatory,  quantitative,  realistic,  objective, 
appropriate,  inclusive,  independent,  valid,  and  reliable.  An  MOE  associates  proper  units 
of  measurement  and  includes  limits  on  the  range  of  the  measurement  as  appropriate. 
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Table  34  Example  interoperability  character  framework 
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